Guide to writing output filters

There are a number of common pitfalls encountered when writing output filters; this page aims to document best practice for authors of new or existing filters.

This document is applicable to both version 2.0 and version 2.2 of the Apache HTTP Server; it specifically targets RESOURCE

-level filters though some advice is generic to all types of filter.

library which provides the bucket brigade interface), but modules are free to define their own types.

A filter can tell whether a bucket represents either data or metadata using the APRBUCKETIS_METADATA

macro. Generally, all metadata buckets should be passed down the filter chain by an output filter. Filters may transform, delete, and insert data buckets as appropriate.

bucket indicates that the end of the response has been reached and no further buckets need be processed. A FLUSH

bucket indicates that the filter should flush any buffered buckets (if applicable) down the filter chain immediately.

buckets are sent when the content generator (or an upstream filter) knows that there may be a delay before more content can be sent. By passing FLUSH

buckets down the filter chain immediately, filters ensure that the client is not kept waiting for pending data longer than necessary.Filters can create FLUSH

buckets and pass these down the filter chain if desired. Generating FLUSH

buckets unnecessarily, or too frequently, can harm network utilisation since it may force large numbers of small packets to be sent, rather than a small number of larger packets. The section on Non-blocking bucket reads covers a case where filters are encouraged to generate FLUSH

HEAP FLUSH FILE EOS

This shows a bucket brigade which may be passed to a filter; it contains two metadata buckets (FLUSH

For any given request, an output filter might be invoked only once and be given a single brigade representing the entire response. It is also possible that the number of times a filter is invoked for a single response is proportional to the size of the content being filtered, with the filter being passed a brigade containing a single bucket each time. Filters must operate correctly in either case.

An output filter can distinguish the final invocation for a given response by the presence of an EOS

bucket in the brigade. Any buckets in the brigade after an EOS should be ignored.

An output filter should never pass an empty brigade down the filter chain. To be defensive, filters should be prepared to accept an empty brigade, and should return success without passing this brigade on down the filter chain. The handling of an empty brigade should have no side effects (such as changing any state private to the filter).

aprstatust dummyfilter(apfiltert f, aprbucket_brigade bb)

{ if (APRBRIGADEEMPTY(bb)) { return APR_SUCCESS; } ....

. The list sentinel is in fact not a valid bucket structure; any attempt to call normal bucket functions (such as aprbucketread

) on the sentinel will have undefined behaviour (i.e. will crash the process).

There are a variety of functions and macros for traversing and manipulating bucket brigades; see the apr_bucket.h header for complete coverage. Commonly used macros include:

structure itself is allocated out of a pool, so if a filter creates a new brigade, it must ensure that memory use is correctly bounded. A filter which allocates a new brigade out of the request pool (r->pool

) on every invocation, for example, will fall foul of the warning above concerning memory use. Such a filter should instead create a brigade on the first invocation per request, and store that brigade in its state structure.

It is generally never advisable to use aprbrigadedestroy

to "destroy" a brigade unless you know for certain that the brigade will never be used again, even then, it should be used rarely. The memory used by the brigade structure will not be released by calling this function (since it comes from a pool), but the associated pool cleanup is unregistered. Using aprbrigadedestroy

can in fact cause memory leaks; if a "destroyed" brigade contains buckets when its containing pool is destroyed, those buckets will not be immediately destroyed.

In general, filters should use aprbrigadecleanup

When dealing with non-metadata buckets, it is important to understand that the "apr_bucket *

" object is an abstract representation of data:

field is set to the value (aprsizet)-1

bucket type have an indeterminate length; they represent the output from a pipe.FILE

bucket type, for example, represents data stored in a file on disk.Filters read the data from a bucket using the aprbucketread

function. When this function is invoked, the bucket may morph into a different bucket type, and may also insert a new bucket into the bucket brigade. This must happen for buckets which represent data not mapped into memory.

To give an example; consider a bucket brigade containing a single FILE

bucket to represent that data, and return the data to the caller. It also inserts a new FILE

call, the brigade looks like:

HEAP(8K) FILE(8K-24K)

Taking an example which loops through the entire brigade as follows:

aprbucket e = APRBRIGADEFIRST(bb); const char data; aprsizet len; while (e != APRBRIGADESENTINEL(bb)) { aprbucketread(e, &data, &length, APRBLOCKREAD); e = APRBUCKETNEXT(e); } return appass_brigade(bb);

In contrast, the implementation below will consume a fixed amount of memory to filter any brigade; a temporary brigade is needed and must be allocated only once per response, see the Maintaining state section.

aprbucket e; const char data; aprsizet len; while ((e = APRBRIGADEFIRST(bb)) != APRBRIGADESENTINEL(bb)) { rv = aprbucketread(e, &data, &length, APRBLOCKREAD); if (rv) ...; / Remove bucket e from bb. / APRBUCKETREMOVE(e); / Insert it into temporary brigade. / APRBRIGADEINSERTHEAD(tmpbb, e); / Pass brigade downstream. / rv = appassbrigade(f->next, tmpbb); if (rv) ...; aprbrigadecleanup(tmpbb); }

struct dummystate { aprbucketbrigade tmpbb; int filterstate; .... }; aprstatust dummyfilter(apfiltert f, aprbucketbrigade *bb) { struct dummystate state; state = f->ctx; if (state == NULL) { / First invocation for this response: initialise state structure. / f->ctx = state = apr_palloc(sizeof state, f->r->pool); state->tmpbb = aprbrigadecreate(f->r->pool, f->c->bucketalloc); state->filterstate = ...; } ...

state structure), those buckets must be set aside. This is necessary because some bucket types provide buckets which represent temporary resources (such as stack memory) which will fall out of scope as soon as the filter chain completes processing the brigade.

To setaside a bucket, the aprbucketsetaside

function can be called. Not all bucket types can be setaside, but if successful, the bucket will have morphed to ensure it has a lifetime at least as long as the pool given as an argument to the aprbucketsetaside

function can be used, which will move all the buckets into a separate brigade containing buckets with a lifetime as long as the given pool argument. This function must be used with care, taking into account the following points:

guarantees that all the buckets in the returned brigade will represent data mapped into memory. If given an input brigade containing, for example, a PIPE

will consume an arbitrary amount of memory to store the entire output of the pipe.apsavebrigade

reads from buckets which cannot be setaside, it will always perform blocking reads, removing the opportunity to use apsavebrigade

" (destination) brigade parameter, the function will create a new brigade, which may cause memory use to be proportional to content size as described in the The aprbucketread

This mode of operation ensures that any filters further down the filter chain will flush any buffered buckets if a slow content source is being used.

A CGI script is an example of a slow content source which is implemented as a bucket type.

buckets which represent the output from a CGI script; reading from such a bucket will block when waiting for the CGI script to produce more output.

aprbucket e; aprreadtypee mode = APRNONBLOCKREAD; while ((e = APRBRIGADEFIRST(bb)) != APRBRIGADESENTINEL(bb)) { aprstatust rv; rv = aprbucketread(e, &data, &length, mode); if (rv == APREAGAIN && mode == APRNONBLOCKREAD) { / Pass down a brigade containing a flush bucket: / APRBRIGADEINSERTTAIL(tmpbb, aprbucketflushcreate(...)); rv = appassbrigade(f->next, tmpbb); aprbrigadecleanup(tmpbb); if (rv != APRSUCCESS) return rv; / Retry, using a blocking read. / mode = APRBLOCKREAD; continue; } else if (rv != APRSUCCESS) { / handle errors / } / Next time, try a non-blocking read first. */ mode = APRNONBLOCK_READ; ... }

In summary, here is a set of rules for all output filters to follow:

bucket down the filter chain if the read blocks, before retrying with a blocking read.Available Languages: en

Modules | Directives | FAQ | Glossary | Sitemap

Apache HTTP Server Version 2.4