From Crypto++ Wiki
Jump to: navigation, search

Pipelining is a paradigm used by Crypto++ which allows data to flow from a source to a sink. As data flows, the data will encounter filters which transform the data in some way en route to the sink. The motivation for the design was the Unix pipe system, so a Crypto++ pipeline has very close analogies to a Unix command.

For example, consider a set of commands to Base64 encode a file and then save the encoding to a second file:

cat filename | base64 > filename.b64

The Crypto++ pipeline to accomplish the same would be:

FileSource f(filename, new Base64Encoder(new FileSink(filename + ".b64")));

Generalizing the pipeline would be:

Source s(source, new Filter(new Filter(new Sink(destination))));

Sometimes you want to process all the data at once. When this is the case, the source provides a parameter called pumAll that indicates it. Other times you want to block or chunk the data in discrete pieces, so you would specify false as pumAll. After blocking, you would Pump the data from the source into a filter.


The Annotated Class Reference includes documentation for the Source interface and Sink interface.

All filters inherit from BufferedTransformation. With respect to BufferedTransformations, intermediate objects of interest are:

With respect to Filters, objects of interest are:

Generally, a user defined filter will derive from class Filter. Wei Dai recommends examining SignerFilter in filters.h for a filter example.


Object ownership is an important detail in a pipeline. According to "Important Usage Notes" in ReadMe.txt:

If a constructor for A takes a pointer to an object B (except primitive types such as int and char), then A owns B and will delete B at A's destruction. If a constructor for A takes a reference to an object B, then the caller retains ownership of B and should not destroy it until A no longer needs it.

That means filters created with new are owned by the outer or encompassing object and will be destroyed by that object when no longer needed. If you need an object to persist, then you should use a Redirector.

Sample Program

The following program transfers data from the first string to the second string. Though not very useful, it is the simplest demonstration of pipelining. An important reminder (from the ReadMe) is that the StringSource, which takes a pointer to the StringSink, owns the StringSink. So the StringSource will delete the StringSink when the StringSource destructor is invoked.

string s1 = "Pipeline", s2;

StringSource ss( s1, new StringSink( s2 ) );

cout << "s1: " << s1 << endl;
cout << "s2: " << s2 << endl;

The following program uses an AutoSeededRandomPool to generate an AES key. The key is hex encoded and then printed.

AutoSeededRandomPool prng;

prng.GenerateBlock( key.data(), key.size() );

string encoded;

StringSource ss( key.data(), key.size(), true,
    new HexEncoder(
        new StringSink( encoded )
    ) // HexEncoder
); // StringSource

cout << "key: " << encoded << endl;

At times, a result is required from an intermediate object that is participating in a pipeline. Most notably is the DecodingResult from a HashVerificationFilter. In this situation, we do not want the filter to own the object and attempt to destroy it. To accomplish the goal, a Redirector would be used. Notice that the Redirector takes a reference to an object, and not a pointer to an object.

CCM< AES, TAG_SIZE >::Decryption d;
d.SetKeyWithIV( key, key.size(), iv, sizeof(iv) );

AuthenticatedDecryptionFilter df( d,
    new StringSink( recovered )
); // AuthenticatedDecryptionFilter

// Cipher text includes the MAC tag
StringSource ss( cipher, true,
    new Redirector( df )
); // StringSource

// If the object does not throw, here's the only
//  opportunity to check the data's integrity
bool b = df.GetLastResult();

if( true == b ) {
    cout << recovered << endl;

One topic that comes up on occasion is skip'ing source bytes. Skip is part of the Filter interface, and it works on the output buffer (more precisely, the AttachedTransformation). To Skip bytes on a Source, then use Pump and discard the bytes by using a NULL AttachedTransformation. Also see Skip'ing on a Source does not work as expected on Stack Overflow and Issue 248: Skip'ing on a Source does not work.

int main(int argc, char* argv[])
  string str1, str2;
  HexEncoder enc(new StringSink(str1));
  for(unsigned int i=0; i < 32; i++)

  cout << "str1: " << str1 <<endl;

  // 'ss' has a NULL AttachedTransformation()
  StringSource ss(str1, false);

  // Attach the real filter chain to 'ss'
  ss.Attach(new StringSink(str2));

  cout << "str2: " << str2 << endl;

  return 0;