dns.c: Asynchronous DNS and SPF Resolver

description

A non-blocking DNS resolver library in a single .c file.

Asynchronous SPF resolver, without caveats—no threading, no forking, no library dependencies.

todo

Write documentation.

Finish EDNS0 support.

Polish address info smart queries (fallback to direct A query when MX query fails, for instance).

Refactor smart query recursion in core resolver to leverage interfaces added for smart queries in the address info API.

Make sure we do the Right Thing with error responses and other wierdness.

Support DNSSEC.

Don't bail on network errors like ECONNREFUSED, ECONNRESET, EHOSTUNREACH, etc. Currently these errors short-circuit resolution, just like ENOMEM, rather than causing resolution to move on to the next nameserver.

news

2015-06-30

Fix potential uninitialized read in nameserver iteration code caught by clang static analyzer.

Tag rel-20150630 (7bafc1cbadab2d67c0502686c6ec860d87524656).

2015-06-12

Fix the return type of acquire and release methods so that the type is the same as the value type of dns_atomic_t.

Fix some signedness issues on 32-bit platforms.

Tag rel-20150612 (e9d62fc89caa2496a0ad1ed84d40bda94f9ccbb8).

2015-06-10

Add dns_res_submit2, which takes the length of the qname. Original path from anonymous contributor.

Add support for GCC's __atomic builtins for thread-safe reference count manipulation. Unlike the Intel/GCC __sync API, dns.c can detect lock-free atomic support for the __atomic API using the preprocessor. Rather than relying on automated detection of atomic APIs, two new macros, DNS_ATOMIC_FETCH_ADD and DNS_ATOMIC_FETCH_SUB, may be optionally defined at build time and which should expand to so-named expression. Additionally, the autoconf-like HAVE___ATOMIC_FETCH_ADD and HAVE___ATOMIC_FETCH_SUB maybe be defined to control whether the GCC __atomic builtins are used. Note that unlike the usual autoconf style, the code checks for 0 or 1, not defined or undefined. (Undefined macros resolve to 0, anyhow, in preprocessor arithmetic expressions.)

2015-06-10

Don't require a resolver object when AI_NUMERICHOST is specified for dns_ai_open.

Add and use DNS_ENONAME and DNS_EFAIL, similar to POSIX EAI_NONAME an EAI_FAIL. Now if no addresses were found dns_ai_nextent will return either DNS_ENONAME (if the last error was DNS_RC_NOERROR or DNS_RC_NXDOMAIN) or DNS_EFAIL (all other conditions). Otherwise dns_ai_nextent still returns ENOENT to mean "no more addresses".

2014-08-15

Fix use of SO_NOSIGPIPE, which was only set on datagram sockets rather than stream sockets.

2014-08-14

Add dns_res_sethints, allowing hints to be swapped out without destroying an object.

In tcp:only mode try to reuse existing TCP stream. Note that the resolver needs to be fixed so that resolution doesn't bail on network errors like ECONNREFUSED, ECONNRESET, EHOSTUNREACH, etc. Currently these errors short-circuit resolution, just like ENOMEM.

2014-08-11

Fix issue where when in stub mode we generated and returned a SERVFAIL answer packet when the more sensible thing was to return NXDOMAIN. This was because on NXDOMAIN the resolver would return to the domain search algorithm and also finish running through the other lookup methods. At the end of the process if nothing was found there was no NXDOMAIN answer packet cached to return. Now in stub mode the first from-the-wire answer packet is cached and is returned to the caller if no other answer is found and no other intermediate errors occured.

Note that recursive resolvers don't normally have this problem because eventually they'll get an NXDOMAIN with the authoritative bit set. The resolver will short-circuit on the authoritative bit. However, it's technically possible that we never get an answer with the authoritative bit set (maybe there's a proxy) and the resolver keeps churning until it runs out of options. Then it would also generate a SERVFAIL packet. That might not be the most sensible thing, although it makes more sense in the recursive case than in the stub case.

2014-06-17

Update root nameserver hints. One old root address in particular was no longer available and would cause slow lookups in recursive mode using the built-in hints. In a real product using recursive lookups you'd want to query the root hints dynamically and then maintain your own hints object.

Fix SRV parsing bug which caused some records to fail with DNS_EILLEGAL.

Don't abort on assert when calling dns_res_check after the answer has already been fetch. Instead, return an explicit error, DNS_EFETCHED.

Replace some asserts with error returns.

Add dns_res_timeout and dns_ai_timeout.

Call fd close callback for deferred descriptor closes.

2013-12-20

Explicitly assign NULL to some free'd member pointers to appease static analyzers like Coverity, which are blind to the subsequent memset operation which resets those members and several others. Reported by Enlightenment team.

2013-06-19

Fix bug in dns_d_trim.

2012-10-03

Fix the core resolver to actually iterate the resconf->search list.

2012-08-05

Fix some conformance niggles with signal mask manipulation in dns.c.

Teach socket.c about F_SETNOSIGPIPE and O_NOSIGPIPE. Refactor and consolidate the way simple boolean descriptor options are handled.

Forgot to mention a FreeBSD regression occurred with the July 10 commit to fix Solaris endianess, released as rel-20120711. It was remedied the day after tagging the release. I still think it was the better decision to directly expose the packet header as a bitfield, like in the ancient BSD API. One of the downsides was crap like this happening.

Tag rel-20120805 (f548b3b830b33151fe3c7724d620fb8082c6ee7c).

2012-08-03

Made another go at writing the user guide. Ended up spinning my wheels trying to devise an API reference layout. That was a fool's errand. The header should suffice as an API reference. What's needed is an overview of usage, a description of the most relevant interfaces, and annotated examples. Next time....

2012-08-02

Reimplement SIGPIPE suppression in dns.c using a signal masking technique described by Tomash Brechko.

Add nsswitch.conf parser and attempt to parse /etc/nsswitch.conf automatically when instantiating a resolver with dns_res_stub. This support was a day long chore given the status=action critera syntax and sadly adds about 500 more lines. No other configuration code was modified or perturbed to minimize regression potential. In particular, all the resolv.conf code remains as-is. But in the future some of the resolv.conf code will be refactored to make use of newer routines to claw back a lower SLoC count.

Although the nsswitch.conf criteria are parsed and grokked, the criteria are not yet obeyed in the core resolver. Presently the criteria are just transformed to a compact string form and written out to dns_resolv_conf->lookup. A hosts entry like "dns [UNAVAIL=return] files" is transformed to "bURf" but behaves like "bf".

Add a form feed character between the library code and the regression utility command code. Now the 1000+ lines of the utility can be excluded with:

cat dns.c | sed -e "/$(printf "\\f")/,//d"

2012-07-11

Ignore SIGPIPE when sending queries over TCP. Fix byte order detection on Solaris so packet header bit fields are arranged properly. Emit tcp option parameter when printing dns_resolv_conf structure.

Add new .fd_nosigpipe option to socket.c, and enable it by default.

SIGPIPE code optimizes the case where SO_NOSIGPIPE or MSG_NOSIGNAL is available.

Tag rel-20120711 (1540dd19c475efe0dc967ad5ac91440d7245f35a).

2012-06-21

Tag rel-20120620 (8048cfe37800b58a0dbd44fc1cc482f24d3eef1e).

2012-06-20

Merge some updates and fixes to contrib/socket.c, including a fix for a possible stall condition caused by a successful connect() state transition masking a write attempt.

2012-06-19

Remove __attribute__((unused)) annotation on goto label because it chokes Solaris Studio.

Exclude FreeBSD and Solaris from the feature macro tarpit, and use _NETBSD_SOURCE for NetBSD. All of this will be revisited and refactored at a later date now that I have a better understanding of the behavior of feature macros on different platforms.

2012-06-18

Fix a few inet_pton usage bugs which improperly returned errno. Consolidate error checking inside new dns_pton/dns_ntop routines, and add new error code for bad textual addresses.

2012-06-16

Eliminate all known compiler warnings in clang and GCC by refactoring some code, and annotating other code using attributes and pragmas.

2011-01-14

Fix bug where dns_srv_print() failed to print a trailing NUL character.

Tag rel-20110114 (a4b31ba8fb6ffba5ad095cf0872ce4944e0d023e).

2010-08-15

Tag rel-20100813 (b03081b6c0c45f109fa671e182b797df5cabdf3d).

2010-08-15

Add socket.c into contrib/. I've been using and developing this for a year or so within other projects. Conceptually it's a simple wrapper around the BSD sockets API that handles DNS lookups and SSL negotiation transparently and asynchronously.

After creating a socket with so_open(), the caller sets the appropriate modes, e.g. so_connect() and so_starttls(). (SSL/TLS mode can be initiated at any time.) The caller can then wait for them to complete (they'll return EAGAIN if necessary), or more conveniently ignore the result and move directly to attempting reads and writes. Errors from the previous states, including EAGAIN, will be returned on any read or write attempts until those states are complete.

Use so_pollfd() and so_events() just like the dns.c counterparts. They return, respectively, the appropriate descriptor and event flags depending on which internal state is executing. The application need only worry about issuing logical read/write operations.

2010-08-13

Use ai->qname if AI_CANONNAME flag is set and ai->cname is empty, such as is the case with a numeric host. OpenBSD's getaddrinfo() will set ai_canonname to the numeric address too.

2010-08-03

Fix memory leak. Partially setup query frames were not being reset because the frame stack pointer was not yet updated. Now all frames are reset regardless.

2010-07-17

Add a binary data formatting utility to make it easier to construct packets and compressed domain names for regression testing. The syntax is simple: everything is a literal, except (1) whitespace is discarded; (2) blackslash precedes an encoded sequence; and (3) hash begins a comment till the end-of-line. An encoded sequence is either a sequence of hexadecimal graph characters which outputs the corresponding octets padded or truncated according to the optional width specifier, or one of the traditional two-glyph control character sequences—e.g. \n. Otherwise, blackslash generates the second glyph. \# and \\ output a literal hash and blackslash, respectively.

The format for google.com as a compressed name is \06 google \03 com \00.

To output N bits use the sequence width specifier, like \f:4, which outputs four bits. The width specifier is in decimal notation. Conversely, \ff:24, outputs three octets, the first two all-bits-zero.

Currently a newline flushes the bitstream, padding to an 8-bit boundary if necessary. This may change if it turns out to be more a headache than a feature.

2010-07-14

Fix MinGW build by adding internal dns_strsep() and DNS_ETIMEDOUT.

2010-07-14

Tag rel-20100709 (464fc64c3cc7ab78efe71226d3eeb09f4bdbead2).

2010-07-09

Audited use of dns_d_expand(). Fail any operation that necessarily depends on untrucated expansion of a domain name. For example, don't grep for records based on search criteria where one of our keys is a truncated name. But preserve those cases where truncation is benign (e.g. debug printing routines), or where execution can progress by skipping whatever operation needs the expansion. This preserves the ability to deconstruct technically invalid but perhaps interesting packets. For example, network monitoring or fuzzing.

2010-07-08

Check the validity of the length returned by dns_d_expand() before passing it as the source length to dns_d_cleave() inside of dns_hints_query() else we might do an invalid read. Thanks to Anonymous.

Tag rel-20100708 (d493a0f7d8f1d67ef312a7ca3e142660895b32d8).

NOTE: Forgot to bump DNS_V_REL for the 20100708 release, and because the new stats interface was in the trunk also should have bumped DNS_V_API. Too late now.

2010-06-23

Add statistics interface. Keep track of number of packets and bytes sent and received.

2010-06-21

Accept NULL servname for dns_ai_open().

2010-06-07

Tag rel-20100515 (e4e31154adbdbe521815ee23e4c76ad60019715c).

2010-05-15

Loop over nameservers resolv.conf:options.attempts times.

2010-05-03

Enforce SPF canonical name query limits when running MX and PTR mechanisms.

Add dns_p_study() to learn and store section ranges.

Keep track of section ranges in dns_p_push().

Refactor dns_p_merge() to improve performance on large packets.

2010-04-28

Add dns_p_make() for malloc'ing and initializing a new packet.

Enforce SPF term query limits. Changed the layout of struct spf_limits in anticipation of enforcing limits on MX and PTR host queries.

Fix bug in the VM instruction generator for jump indices of more than 255.

2010-04-23

Tag rel-20100423 (c8c90207643a001a2701e60d15200dcb34e0cf91).

2010-04-23

Switch back to using an anonymous union to align the packet header and data array, rather than casting the data array. This is safer, yet just as easy to read. Breaks C99 compliance, but anonymous unions should be in C1x, so eventually dns.c will be standards compliant again. spf.rl uses anonymous unions anyhow.

Remove -Werror from CFLAGS because even system macros like IN6_IS_ADDR_V4MAPPED make GCC 4.4+ complain about type-punning.

2010-04-17

Added SSHFP support.

2010-04-17

Tag rel-20100416 (5bd9963e693510e485a1f081f6c98a95d84debfe).

2010-04-16

90% passage rate on the OpenSPF test suite. Need to simulate timeouts to pass the TempError tests. Some of the grammar tests are debatable.

2010-04-14

Added OpenSPF YAML test suite processor. Requires libyaml.

2010-04-12

Query SPF or TXT or both to locate a policy.

Tag rel-20100412 (9a52f5ee4eb3deb3994a57b2cb72c3464fece791).

2010-04-11

Added cache.c, a simple query cache, to support the OpenSPF test suite and document use of the new cache API in the core resolver.

2010-04-10

In preparation for supporting the OpenSPF test suite, added a cache API to the DNS resolver code, and a new (c)ache lookup method, analogous to the (b)ind and (f)ile methods. This API allows an application to specify both synchronous and asynchronous cache interfaces.

Added zone.rl, an RFC 1035 master file (aka zone file) format parser.

Removed _pollin and _pollout interfaces in favor of _pollfd and _events.

2010-04-09

Add string parsing for section, class, opcode, and rcode types to support script-language bindings.

2010-04-06

Support bare IP4 terms, which appear to be commonly used in SPF policies.

2010-04-03

Fix bug where term default domain, %{d}, wasn't expanded.

2010-03-24

Set default nameserver to 0.0.0.0 per traditional BSD policy.

2010-03-03

Replace internal _shuffle8 with _shuffle16 to mix full width of record index when using the shuffle rrset iterator.

2010-02-10

Release 20100210 (cb14dea94c0ed41c6c65adfa670ab597a6b08589).

Add dns_itype() complement to dns_strtype().

Add version interfaces: dns_vendor(), dns_v_rel(), dns_v_abi(), dns_v_api().

Add new (struct dns_options) parameter to object instantiators.

Add optional closefd callback to explictly notify app of discarded descriptors.

Support libevent bitmasks from _events() interfaces instead of poll(2) bitmasks.

Add _clear() routines to force closure of queued, discarded descriptors. Alternative to closefd callback, but also optional.

2009-11-28

Delay closure of descriptors so that kqueue(2) and epoll(2) callers have a chance to recognize a state change after installing a persistent event and where sequential descriptors with the same integer value returned from _pollfd() would be ambiguous.

Another interface needs to be added to flush the descriptors. Currently they're only flushed when the socket or resolver object is destroyed. Final solution will consist of one or all of (1) a new routine, perhaps exposing the _closefds() internal; (2) a new option to control automatic flushing; and/or (3) a callback to notify a caller exactly when a particular descriptor is being closed. The latter two will probably require a new options structure other than the resolv_conf structure.

This only changes the behavior of TCP descriptors. The UDP descriptor is retained throughout the lifetime of a socket or resolver object.

Support OpenBSD /etc/resolv.conf tcp option. As on OpenBSD it can be specified as plain "tcp" to force TCP only querying; or one of "tcp:enable", "tcp:only", or "tcp:disable". "tcp:enable" is the default, and "tcp:only" is equivalent to "tcp". "tcp:disable" disables TCP, currently causing a truncated packet to be returned when the resolver would otherwise switch to TCP.

Switch version name schema and roll 20091128 from commit 70a177ebd929e4afd454bc7369ac79e68335f902.

2009-11-04

Support OpenBSD /etc/resolv.conf nameserver port extension; e.g. nameserver [1.2.3.4]:5353. This will be useful when trying to bootstrap the OpenSPF.org test suite.

Add additional SPF VM ops and some premature performance optimizations.

Support C constants from SPF VM assembler; e.g. $AF_INET. See fcrd.spf for examples.

Roll 0.6.1 from commit 5668bd4c87471b7c482e095ffbc95c7da4a6298d.

2009-11-02

Reorganize SPF code so it's easier to read. Some bug fixes to the spf utility.

Add fcrd.spf, example bytecode for doing forward-confirmed reverse DNS the "hard way"; that is, without the built-in fcrd/fcrdx ops, and without the built-in addrinfo/nextent ops (which the fcrd macro op uses when it dynamically generates code to do the confirmation). This helps to show the way to refactoring the VM to support c-ares, without the benefit of the highly useful ancillary routines in dns.c :)

2009-11-01

Add dns_res_events() and dns_res_pollfd() to supplement/replace dns_res_pollin() and dns_res_pollout(). This makes it a tad easier to work with libevent (saves a few lines of code).

Debut asynchronous SPF resolver. WARNING: This is alpha code!

2009-05-28

Fix nasty regression in previous bugfix that broke CNAME chaining in stub mode.

2009-04-17

Fix bug where we didn't fallback from "bind" method to "file" method if the recurse flag was disabled. This solves the annoyance of, for instance, unqualified "localhost" not resolving. But, this calls for closer inspection of the search generator, me thinks.

Search generator may have changed the qname. So, in dns_ai_nextent() canonicalize the qname from the answer, not the qname originally submitted to the resolver.

2009-03-21

Numeric resolution in dns_ai_nextent() did not set the port properly. Fixed.

Add dns_res_stub() for convenience. Returns a resolver configured as a stub resolver, without requiring the user to manually instantiate the proper configuration objects.

Add include guards in dns.h.

2009-03-16

Set feature macros for POSIX/SUSv3 and BSD interfaces in dns.c. This should have the effect of fixing GNU/Linux regression compilation, and hopefully continue to build on BSD systems without issue.

When building against dns.h, one must ensure the definition of POSIX struct addrinfo. In practice, this means that on Linux (with glibc) one should define _POSIX_C_SOURCE or _XOPEN_SOURCE or _GNU_SOURCE, as appropriate. These are not defined in dns.h because of potential side-effects in application code. Alternatively, invoking GCC with -std=gnu99 rather than -std=c99 might work.

2009-03-09

Port to MinGW32 / Win32.

2009-03-08

Added address info API, semantically similar to POSIX getaddrinfo(), but non-blocking, re-entrant, and iterator-based. Does "smart" queries more optimally, since per-record recursion can be postponed (unlike "smart" recursion in the core resolver, which must collect all necessary records to return in the single answer packet).

struct addrino hints = { .ai_flags = AI_CANONNAME, };
struct dns_resolver *res = dns_res_open(...);
int error;

struct dns_addrinfo *ai = dns_ai_open("google.com", "25", DNS_T_MX, &hints, res, &error);
...
struct addrinfo *res;
int error;

switch ((error = dns_ai_nextent(&res, ai))) {
case 0:
/* do something */
case ENOENT:
/* no more */
case EAGAIN:
/* poll on dns_ai_pollin() / dns_ai_pollout() */
default:
/* OOPS! */
}

Fixed SRV record bug. Wrong value for record data length was written out when composing SRV object as packet data.

2009-02-28

Original public release.

usage

The last 1/6 of dns.c contains a full regression suite. The regression binary is the default target of the included Makefile.

Run make dns && ./dns -h for the regression commands and options. The regression tests show how to use the library.

Similarly, spf.rl compiles to a standalone utility which spf.t runs several regression tests against. The utility also includes a built-in assembler for executing ad hoc bytecode in the SPF VM interpreter (see fcrd.spf).

Run make spf && ./spf -h for the commands and options.

NOTICE: Just as event flags can change depending on whether the resolver is waiting for read or write readiness, dns_res_pollfd(), dns_ai_pollfd(), and spf_pollfd() can return different descriptors across invocations. For example, if the underlying resolver needs to switch to TCP to read a truncated answer. If user code doesn't account for this, events registered with your event loop may timeout. (Alternatively, disable TCP querying for a quick fix.) By default dns.c only closes descriptors upon object destruction, otherwise kqueue() or epoll() user state would be corrupted. For immedate closure employ the (struct dns_options).closefd callback.

For more information, until documentation can be written, just read the source code. Comments are sparse but meaningful where they exist.

license

Copyright (c) 2008-2015 William Ahern

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

download

Not currently intended to be versioned as an installable library. All updates are documented in Git.

For convenience, libdns-20150630.tgz. This is not always up-to-date.

source

git clone http://25thandClement.com/~william/projects/libdns.git

Or visit the GitHub mirror

other projects

airctl | bsdauth | cnippets | libarena | libevnet | authldap | streamlocal | libnostd | zoned | dns.c | delegate.c | llrb.h | lpegk | json.c | cqueues | siphash.h | hexdump.c | timeout.c | luapath | luaossl | lunix | phf | runlua | tarsum | prosody-openbsd | AnonNet