HexEncoder

From Crypto++ Wiki
(Redirected from Hex encoding)
Jump to: navigation, search

The HexEncoder encodes bytes into base 16 encoded data. The partner decoder is a HexDecoder.

The HexEncoder takes a pointer to a BufferedTransformation. Because a pointer is taken, the HexEncoder owns the attached transformation, and therefore will destroy it. See ownership for more details.

HexEncoder
Documentation
#include <cryptopp/hex.h>

Construction

HexEncoder(BufferedTransformation *attachment=NULL,
           bool uppercase=true,
           int outputGroupSize=0,
           const std::string &separator=":",
           const std::string &terminator="")

attachment is a BufferedTransformation, such as another filter or sink. If attachment is NULL, then the HexEncoder object will internally accumulate the output byte stream.

uppercase is an output formatting option and determines if output is uppercase or lowercase.

outputGroupSize is an output formatting option and determines the number of hexadecimal digit groups. For example, if outputGroupSize = 4, then an output string is formatted as "FFEE:DDCC:BBAA:9988:7766:5544:3322:1100".

separator is a string used as a delimiter. The default is a colon, and a space (with a grouping of 4) will format the string as "FFEE DDCC BBAA 9988 7766 5544 3322 1100". Encoding a Binary String for C Output shows a slightly more interesting use of the delimiter.

terminator adds a terminator to the output string. If outputGroupSize = 0, then a terminator of 'h' could be used to signify a hexadecimal string: "FFEEDDCCBBAA998877665544332211h".

Examples

The following is a small collection of examples to demonstrate using the HexEncoder.

Encoding a Binary String (Non-Filter)

The following demonstrates decoding a string using Put and Get.

byte decoded[] = { 0xFF, 0xEE, 0xDD, 0xCC, 0xBB, 0xAA, 0x99, 0x88, 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00 };
string encoded;

HexEncoder encoder;
encoder.Put(decoded, sizeof(decoded));
encoder.MessageEnd();

word64 size = encoder.MaxRetrievable();
if(size)
{
    encoded.resize(size);		
    encoder.Get((byte*)encoded.data(), encoded.size());
}

cout << encoded << endl;

A run of the above program produces the following output.

$ ./cryptopp-test.exe
FFEEDDCCBBAA99887766554433221100

Encoding a Binary String (Filter)

Encoding a String (Non-Filter) performed a Put/Get sequence to transform the data. Crypto++ offers filters, which can simplify the process as shown below by taking advantage of Crypto++'s pipeline design.

byte decoded[] = { 0xFF, 0xEE, 0xDD, 0xCC, 0xBB, 0xAA, 0x99, 0x88, 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00 };
string encoded;

StringSource ss(decoded, sizeof(decoded), true,
    new HexEncoder(
        new StringSink(encoded)
    ) // HexEncoder
); // StringSource

cout << encoded << endl;

As with the previous example, a run produces the following output.

$ ./cryptopp-test.exe
FFEEDDCCBBAA99887766554433221100

Encoding a Binary String for C Output

The following produces C array style output. Notice the the encoded string is "0x", and the separator is specified as ", 0x". The separator ensures all element (except the first) have a "0x". To embue the first element we a "0x", we simply set the output string encoded to "0x".

byte decoded[] = { 0xFF, 0xEE, 0xDD, 0xCC, 0xBB, 0xAA, 0x99, 0x88, 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00 };
string encoded = "0x";

StringSource ss(decoded, sizeof(decoded), true,
    new HexEncoder(
        new StringSink(encoded),
        true,   // uppercase
        2,      // grouping
        ", 0x"  // separator
    ) // HexDecoder
); // StringSource

cout << encoded << endl;

Output is as follows.

$ ./cryptopp-test.exe
0xFF, 0xEE, 0xDD, 0xCC, 0xBB, 0xAA, 0x99, 0x88, 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00

Attaching a BufferedTransformation

Sometimes its advantageous to attach (or change an attached) BufferedTransformation on the fly. The code below attaches a StringSink at runtime.

byte decoded[] = { 0xFF, 0xEE, 0xDD, 0xCC, 0xBB, 0xAA, 0x99, 0x88, 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00 };
string encoded;
   
HexEncoder encoder;
 
encoder.Attach( new StringSink( encoded ) );
encoder.Put( decoded, sizeof(decoded) );
encoder.MessageEnd();

Attach returns the previous attached transformation. The caller is responsible for deleting the previous filter if its non-NULL. If you want to attach a new transformation and delete the current one, then use the Detach method. Detach will free the currently attached filter, and add the new transformation.

Scripting and Strings

On occasion, the mailing list will receive questions on cross-validation. For example, see AES CTR Chiper. Different output between PHP-mcrypt and Crypto++. In the question, PHP-mcrypt strings are used as follows:

$key = "1234567890123456789012345678901234567890123456789012345678901234";
$key = pack("H".strlen($key), $key);
$iv = "1111111111222222222233333333334444444444555555555566666666667777";
$iv = pack("H".strlen($iv), $iv);

One of the easiest ways to avoid typos is via Copy/Paste and a HexDecoder:

string encodedKey = "1234567890123456789012345678901234567890123456789012345678901234";
string encodedIv = "1111111111222222222233333333334444444444555555555566666666667777";
string key, iv;

StringSource ssk(encodedKey, true /*pump all*/,
    new HexDecoder(
        new StringSink(key)
    ) // HexDecoder
); // StringSource

StringSource ssv(encodedIv, true /*pump all*/,
    new HexDecoder(
        new StringSink(iv)
    ) // HexDecoder
); // StringSource

After running the above code, key and iv are hexadecimal (i.e., binary) strings rather than printable (i.e., ASCII) strings. The binary strings can be examined using GDB's x command.

(gdb) p key
$1 = {
  static npos = <optimized out>, 
  _M_dataplus = {
    <std::allocator<char>> = {
      <__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, 
    members of std::basic_string<char>::_Alloc_hider: 
    _M_p = 0x100403c18 "\0224Vx?\0224Vx?\0224Vx?\0224Vx?\0224Vx?\0224Vx?\0224"
  }
}
(gdb) x/16b 0x100403c18
0x100403c18:	0x12	0x34	0x56	0x78	0x90	0x12	0x34	0x56
0x100403c20:	0x78	0x90	0x12	0x34	0x56	0x78	0x90	0x12

(gdb) p iv
$2 = {
  static npos = <optimized out>, 
  _M_dataplus = {
    <std::allocator<char>> = {
      <__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, 
    members of std::basic_string<char>::_Alloc_hider: 
    _M_p = 0x100403cf8 "\021\021\021\021\021\"\"\"\"\"33333DDDDDUUUUUfffffww"
  }
}
(gdb) x/16b 0x100403cf8
0x100403cf8:	0x11	0x11	0x11	0x11	0x11	0x22	0x22	0x22
0x100403d00:	0x22	0x22	0x33	0x33	0x33	0x33	0x33	0x44

Finally, the strings key and iv can be used with encryption or decryption objects as follows.

CTR_Mode< AES >::Encryption enc;
enc.SetKeyWithIV(key.data(), key.size(), iv.data());

Missing Data

Its not uncommon to send data through a pipeline and then have nothing in the sink. This is usually due to the compiler matching the wrong function. For example:

string source = "FF 88 00", destination;
StringSink ss(source,
    new HexDecoder(
        new StringSink(destination)
    ) // HexDecoder
); // StringSink

After the above code executes, destination will likely be empty because the compiler coerces the HexDecoder (the pointer) to a bool (the pumpAll parameter), which leaves the StringSource's attached transformation NULL. The compiler will do so without warning, even with -Wall -Wextra -Wconversion. To resolve the issue, explicitly specify the pumpAll parameter:

string source = "FF 88 00", destination;
StringSink ss(source, true /*pumpAll*/,
    new HexDecoder(
        new StringSink(destination)
    ) // HexDecoder
); // StringSink

Another way data ends up missing is failing to call MessageEnd() when pumping data. For example, the following may not produce expected results:

// The 4-bit nibble will be buffered waiting for another nibble
string source = "FF 88 0", destination;

HexDecoder decoder(new StringSink(destination));
decoder.Put(source.data(), source.size());

// Do something with destination

Be sure to call MessageEnd() when data comes up missing:

string source = "FF 88 0", destination;

HexDecoder decoder(new StringSink(destination));
decoder.Put(source.data(), source.size());
decoder.MessageEnd();

// Do something with destination