OpenCSW (Command Line)

From Crypto++ Wiki
Jump to: navigation, search

OpenCSW is an open source project which provides binary Solaris packages for Solaris 9 through 11 on x86 and Sparc platforms . Crypto++ does not currently have an OpenCSW package but this page will detail the steps to create one.

We tried to create a package following the OpenCSW instructions but we lacked the practical knowledge of OpenCSW for the task. Embarrassingly we were not able to make an existing package once we installed the requisite tools and checked out the source build tree. At this point we decided to retreat and document the Crypto++ side of the process for folks who are skilled with OpenSCW.

The easiest way to build Crypto++ for distros is the unofficial Autotools files. There are six steps to the process and they are the same as other Autotools projects once you fetch the files. The six steps begin at Obtain Source Files and finish at Install the Library

Though Autotools is unofficial it is well supported by the Crypto++ and Debian projects. One of the gaps for Autotools is we don't support a lot of Autoconf options, like --disable-static or --disable-PIC. If you don't want the static library then manually delete it after building the library. If you don't want position independent code then modify configure.ac.

The compiler options, flags and directories used by OpenCSW are listed at 32-bit and 64-bit packages.

A related page is Solaris (Command Line), which details some of the work-arounds required to build the library and pass self tests. Solaris and the Sun compiler are not the easiest platforms to work with at times. The Autotools files have the fixes baked in.

Autotools

The easiest way to build Crypto++ for distros is the unofficial Autotools files. Though Autotools is unofficial it is well supported by the Crypto++ and Debian projects.

The Autotools project keeps in stride with Crypto++ Master. You can work from the tip of master on both Crypto++ and Autotools and things will "just work" for you. Crypto++, Autotools and CMake tag releases so you can also download release zips. The Crypto++ 7.0 example below uses master and checks out the CRYPTOPP_7_0_0 tag.

Autotools does not support a lot of Autoconf options, like --disable-static or --disable-PIC, because distros don't vary much in their build specs. If you don't want the static library then manually delete it after building the library. If you don't want position independent code then modify configure.ac.

Obtain Source Files

There are two sets of source files to obtain. The first is the Crypto++ sources and the second is the Autotools project files.

To obtain the latest Crypto++ source files clone them from Wei Dia's GitHub.

$ git clone https://github.com/weidai11/cryptopp
Cloning into 'cryptopp'...
remote: Counting objects: 18015, done.
remote: Compressing objects: 100% (33/33), done.
...

$ cd cryptopp

The Autotools files are a separate project so they must be cloned separately as shown below.

$ git clone https://github.com/noloader/cryptopp-autotools
$ cd cryptopp-autotools
$ cp configure.ac Makefile.am libcryptopp.pc.in ../
$ cd ..

At this point you should open Makefile.am and verify the library version number around line 140. Make sure it looks reasonable because we often forget to increment it.

## Very important... Version number of the library
libcryptopp_la_LDFLAGS = $(AM_LDFLAGS) -version-info 6:0:0

Bootstrap Autotools

The second step bootstraps the Autotools project and produces a configure file.

$ mkdir m4/
$ autoreconf -f -i
...

libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
...

After autoreconf finishes you will have fodder sprayed throughout the root directory like a feral dog sprays its urine. Autotools is not exactly known for being tidy.

Configure the Library

The third step configures the library with the configure scripts. The project follows GNU coding standards and honors a user's CXX, CXXFLAGS and LDFLAGS. The project does not use C related flags like CC or CFLAGS.

Below is an example of using the SunCC compiler from Developer Studio 12.6. After this steps completes the library will be ready to build. This step also deletes the GNUmakefiles so make just works without surprises. Otherwise you would need extra options like make -f Makefile to build the library with the makefiles you just produced.

$ CXX=/opt/developerstudio12.6/bin/CC ./configure
checking for a BSD-compatible install... /usr/bin/ginstall -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/gmkdir -p
...

After configure completes you will see a summary as shown below.

Auto-configuration complete. A summary of options are below. If
something looks wrong then please modify config.h and please report
it at http://github.com/noloader/cryptopp-autotools.

   Build triplet: i386-pc-solaris2.11
 Compiler target:
Compiler version: CC: Studio 12.6 Sun C++ 5.15 SunOS_i386 2017/05/30

Static library: yes
Shared library: yes

CRYPTOPP_SSE_FLAG: -xarch=sse2
CRYPTOPP_ARIA_FLAG: -xarch=ssse3
CRYPTOPP_BLAKE2_FLAG: -xarch=sse4_2
CRYPTOPP_CHAM_FLAG: -xarch=ssse3
CRYPTOPP_CRC_FLAG: -xarch=sse4_2
CRYPTOPP_LEA_FLAG: -xarch=ssse3
CRYPTOPP_GCM_FLAG: -xarch=aes
CRYPTOPP_AES_FLAG: -xarch=aes
CRYPTOPP_SIMECK_FLAG: -xarch=ssse3
CRYPTOPP_SIMON_FLAG: -xarch=ssse3
CRYPTOPP_SPECK_FLAG: -xarch=ssse3
CRYPTOPP_SM4_FLAG: -xarch=aes

Automake flags (can be overridden by user flags):
AM_CXXFLAGS:  -O2 -template=no%extdef -DCRYPTOPP_DISABLE_SHANI  -D__SSE2__=1 -D__SSSE3__=1 -D__SSE4_1__=1 -D__SSE4_2__=1 -D__AES__=1
AM_LDFLAGS:  -XCClinker -Mcryptopp.mapfile -lnsl -lsocket -xarch=sse2 -xarch=ssse3 -xarch=sse4_1 -xarch=sse4_2 -xarch=aes

User flags (overrides Automake flags on conflict):
CXXFLAGS: -g
LDFLAGS:

CPPFLAGS does not include -DNDEBUG. You should consider building
with NDEBUG defined so an assert does not inadvertently crash your
program and egress sensitive data to an error reporting service
like AppPort, Windows Error Reporting, Crash Reporter, etc.

CXXFLAGS does not include -O3. You should consider building at -O3
to engage compiler vectorizations and enhance performance.

The machine is 64-bit capable but -m64 is not present in CXXFLAGS

Compile Source Files

The fourth step compiles the source files.

$ make -j 5
make  all-am
make[1]: Entering directory '/export/home/build/cryptopp'
cp adhoc.cpp.proto adhoc.cpp
source='cryptlib.cpp' object='libcryptlib_la-cryptlib.lo' libtool=yes \
DEPDIR=.deps depmode=none /bin/sh ./depcomp \
/bin/sh ./libtool  --tag=CXX   --mode=compile /opt/developerstudio12.6/bin/CC -DHAVE_CONFIG_H -I.    -O2 -template=no%extdef -DCRYPTOPP_DISABLE_SHANI  -D__SSE2__=1 -D__SSSE3__=1 -D__SSE4_1__=1 -D__SSE4_2__=1 -D__AES__=1  -g -c -o libcryptlib_la-cryptlib.lo `test -f 'cryptlib.cpp' || echo './'`cryptlib.cpp
source='cpu.cpp' object='libcpu_la-cpu.lo' libtool=yes \
DEPDIR=.deps depmode=none /bin/sh ./depcomp \
/bin/sh ./libtool  --tag=CXX   --mode=compile /opt/developerstudio12.6/bin/CC -DHAVE_CONFIG_H -I.    -O2 -template=no%extdef -DCRYPTOPP_DISABLE_SHANI  -D__SSE2__=1 -D__SSSE3__=1 -D__SSE4_1__=1 -D__SSE4_2__=1 -D__AES__=1  -g -c -o libcpu_la-cpu.lo `test -f 'cpu.cpp' || echo './'`cpu.cpp
...

Test the Library

There are several artifacts created by the make all recipe. They are the shared object, the static library, the cryptest program called cryptest and a special cryptest program called cryptestcwd. The two test programs exist because of a small design issue we have not worked around.

The difference between cryptest and cryptestcwd is, cryptest looks for datafiles in ${prefix}/share while cryptestcwd looks for datafiles in $PWD. You run cryptestcwd to test the library in-place, and you use cryptest once the library has been installed. The make install recipe does not install cryptestcwd.

Since you want to test the library before staging or installation you run the cryptestcwd program as shown below.

$ ./cryptestcwd v
Using seed: 1532473756

Testing Settings...

passed:  Your machine is little endian.
passed:  Aligned data access.
passed:  sizeof(byte) == 1
passed:  sizeof(word16) == 2
passed:  sizeof(word32) == 4
passed:  sizeof(word64) == 8
passed:  sizeof(hword) == 2, sizeof(word) == 4, sizeof(dword) == 8
passed:  cacheLineSize == 64
hasSSE2 == 1, hasSSSE3 == 1, hasSSE4.1 == 1, hasSSE4.2 == 1, hasAVX == 0, hasAVX2 == 0, hasAESNI == 1, hasCLMUL == 1, hasRDRAND == 0, hasRDSEED == 0, hasSHA == 0, isP4 == 0

Testing operating system provided blocking random number generator...

passed:  it took 0 seconds to generate 16 bytes
passed:  16 generated bytes compressed to 19 bytes by DEFLATE
passed:  GenerateWord32 and Crop
...

A second, more comprehensive test is available that runs test vectors as shown below.

$ ./cryptestcwd tv all
Using seed: 1532473795

Testing FileList algorithm all.txt collection.

Testing SymmetricCipher algorithm TEA/ECB.
................................................................
Testing SymmetricCipher algorithm XTEA/ECB.
.................................................................
Testing SymmetricCipher algorithm WAKE-OFB-LE.
.
Testing SymmetricCipher algorithm WAKE-OFB-BE.
...

Install the Library

The final step in the process is installing the library. Installation is accomplished with a make install.

$ sudo make install
make[1]: Entering directory '/export/home/build/cryptopp'
test -z "/usr/local/lib" || /usr/bin/gmkdir -p "/usr/local/lib"
 /bin/sh ./libtool   --mode=install /usr/bin/ginstall -c   libcryptopp.la '/usr/local/lib'
libtool: install: /usr/bin/ginstall -c .libs/libcryptopp.so.7.0.1 /usr/local/lib/libcryptopp.so.7.0.1
libtool: install: (cd /usr/local/lib && { ln -s -f libcryptopp.so.7.0.1 libcryptopp.so.7 || { rm -f libcryptopp.so.7 && ln -s libcryptopp.so.7.0.1 libcryptopp.so.7; }; })
libtool: install: (cd /usr/local/lib && { ln -s -f libcryptopp.so.7.0.1 libcryptopp.so || { rm -f libcryptopp.so && ln -s libcryptopp.so.7.0.1 libcryptopp.so; }; })
libtool: install: chmod +x /usr/local/lib/libcryptopp.so.7.0.1
...

The install recipe places artifacts as follows. Also see Variables for Installation Directories in the GNU manual.

Artifact Location Comments
Header files ${includedir}/cryptopp Library header files. Test header files are not included.
Libraries ${libdir} Named libcryptopp.{a|so}. Soft links to versioned shared object.
Test program ${exec_prefix}/bin cryptest utility and test program.
Data files ${datadir}/cryptopp Library data files. *.data and *.txt files.

Documentation

The library uses Doxygen for its online manual at https://www.cryptopp.com/docs/ref/. The project does not use man pages. If you want to build the documents then Doxygen must be installed.

As of Commit 789a4a2cb48d from July 2018 you can build the docs using Autotools. However, the install recipe does not install them because we don't know how to craft the install rule that uses ${pkghtmldir}. You will have to manually copy the local html-doc folder to ${pkghtmldir}. Also see How to conditionally install documentation to pkghtmldir using Automake on Stack Overflow.

You can also use the GNUmakefile to build the docs as shown below. You may need to checkout the GNUmakefile if it was deleted by configure.

$ make -f GNUmakefile docs
$ make docs
doxygen Doxyfile -d CRYPTOPP_DOXYGEN_PROCESSING
Notice: Output directory `html-docs' does not exist. I have created it for you.
Searching for include files...
Searching for files in directory /home/build/cryptopp-doc
Searching for example files...
Searching for files in directory /home/build/cryptopp-doc
Searching for images...
Searching for dot files...
Searching for msc files...
Searching for dia files...
...

The output of the process is a directory named ref/ with an index.html. The ref/ is also zipped up and named CryptoPPRef.zip. The names ref/ and CryptoPPRef.zip are convenience items for the project and match the layout of directories on the web server.

List Source Files

The library ships with a GNUmakefile to build the library. The GNUmakefile has recipes not present in the Autotools project. One recipe is make sources. The recipe lists the headers and source files used for the library and test program as shown below.

You could use this recipe to ensure the cryptopp-dev package included all the headers for programming with the library. In fact we use it for the Autotools project when building the file lists for targets.

$ make -f GNUmakefile sources
***** Library sources *****
cryptlib.cpp cpu.cpp integer.cpp 3way.cpp adler32.cpp algebra.cpp algparam.cpp a
rc4.cpp aria-simd.cpp aria.cpp ariatab.cpp asn.cpp authenc.cpp base32.cpp base64
.cpp basecode.cpp bfinit.cpp blake2-simd.cpp blake2.cpp blowfish.cpp blumshub.cp
p camellia.cpp cast.cpp casts.cpp cbcmac.cpp ccm.cpp chacha.cpp cham-simd.cpp ch
am.cpp channels.cpp cmac.cpp crc-simd.cpp crc.cpp default.cpp des.cpp dessp.cpp
dh.cpp dh2.cpp dll.cpp dsa.cpp eax.cpp ec2n.cpp eccrypto.cpp ecp.cpp elgamal.cpp
 emsa2.cpp eprecomp.cpp esign.cpp files.cpp filters.cpp fips140.cpp fipstest.cpp
 gcm-simd.cpp gcm.cpp gf256.cpp gf2_32.cpp gf2n.cpp gfpcrypt.cpp gost.cpp gzip.c
pp hc128.cpp hc256.cpp hex.cpp hight.cpp hmac.cpp hrtimer.cpp ida.cpp idea.cpp i
terhash.cpp kalyna.cpp kalynatab.cpp keccak.cpp lea-simd.cpp lea.cpp luc.cpp mar
s.cpp marss.cpp md2.cpp md4.cpp md5.cpp misc.cpp modes.cpp mqueue.cpp mqv.cpp nb
theory.cpp neon-simd.cpp network.cpp oaep.cpp ospstore.cpp osrng.cpp padlkrng.cp
p panama.cpp pkcspad.cpp poly1305.cpp polynomi.cpp ppc-simd.cpp pssr.cpp pubkey.
cpp queue.cpp rabbit.cpp rabin.cpp randpool.cpp rc2.cpp rc5.cpp rc6.cpp rdrand.c
pp rdtables.cpp rijndael-simd.cpp rijndael.cpp ripemd.cpp rng.cpp rsa.cpp rw.cpp
 safer.cpp salsa.cpp scrypt.cpp seal.cpp seed.cpp serpent.cpp sha-simd.cpp sha.c
pp sha3.cpp shacal2-simd.cpp shacal2.cpp shark.cpp sharkbox.cpp simeck-simd.cpp
simeck.cpp simon-simd.cpp simon.cpp skipjack.cpp sm3.cpp sm4-simd.cpp sm4.cpp so
cketft.cpp sosemanuk.cpp sparx.cpp speck-simd.cpp speck.cpp square.cpp squaretb.
cpp sse-simd.cpp strciphr.cpp tea.cpp tftables.cpp threefish.cpp tiger.cpp tiger
tab.cpp trdlocal.cpp ttmac.cpp tweetnacl.cpp twofish.cpp vmac.cpp wait.cpp wake.
cpp whrlpool.cpp xtr.cpp xtrcrypt.cpp zdeflate.cpp zinflate.cpp zlib.cpp

***** Library headers *****
3way.h adler32.h adv-simd.h aes-armv4.h aes.h algebra.h algparam.h androidpay.h
arc4.h argnames.h aria.h asn.h authenc.h base32.h base64.h basecode.h blake2.h b
lowfish.h blumshub.h camellia.h cast.h cbcmac.h ccm.h chacha.h cham.h channels.h
 cmac.h config.h cpu.h crc.h cryptlib.h default.h des.h dh.h dh2.h dll.h dmac.h
drbg.h dsa.h eax.h ec2n.h eccrypto.h ecp.h ecpoint.h elgamal.h emsa2.h eprecomp.
h equihash.h esign.h fhmqv.h files.h filters.h fips140.h fltrimpl.h gcm.h gf256.
h gf2_32.h gf2n.h gfpcrypt.h gost.h gzip.h hashfwd.h hc128.h hc256.h hex.h hight
.h hkdf.h hmac.h hmqv.h hrtimer.h ida.h idea.h integer.h iterhash.h kalyna.h kec
cak.h lea.h lubyrack.h luc.h mars.h md2.h md4.h md5.h mdc.h mersenne.h misc.h mo
darith.h modes.h modexppc.h mqueue.h mqv.h naclite.h nbtheory.h network.h nr.h o
aep.h oids.h ospstore.h osrng.h ossig.h padlkrng.h panama.h pch.h pkcspad.h poly
1305.h polynomi.h ppc-simd.h pssr.h pubkey.h pwdbased.h queue.h rabbit.h rabin.h
 randpool.h rc2.h rc5.h rc6.h rdrand.h rijndael.h ripemd.h rng.h rsa.h rw.h safe
r.h salsa.h scrypt.h seal.h secblock.h seckey.h seed.h serpent.h serpentp.h sha.
h sha3.h shacal2.h shark.h simeck.h simon.h simple.h siphash.h skipjack.h sm3.h
sm4.h smartptr.h socketft.h sosemanuk.h sparx.h speck.h square.h stdcpp.h strcip
hr.h tea.h threefish.h tiger.h trap.h trdlocal.h trunhash.h ttmac.h tweetnacl.h
twofish.h vmac.h wait.h wake.h whrlpool.h winpipes.h words.h xtr.h xtrcrypt.h zd
eflate.h zinflate.h zlib.h

***** Test sources *****
adhoc.cpp test.cpp bench1.cpp bench2.cpp validat0.cpp validat1.cpp validat2.cpp
validat3.cpp validat4.cpp datatest.cpp regtest1.cpp regtest2.cpp regtest3.cpp dl
ltest.cpp fipsalgt.cpp

***** Test headers *****
bench.h factory.h validate.h

make: Nothing to be done for 'sources'.

Library Version

We don't always remember to bump the library version number when working with the Autotools project. If you use the project files then please double check the version in Makefile.am around line 140.

## Very important... Version number of the library
libcryptopp_la_LDFLAGS = $(AM_LDFLAGS) -version-info 7:0:0

Crypto++ 7.0

Crypto++ 7.0 is the latest version of the library as of this writing. Here are the steps that fold the proceeding sections into one mostly copy/paste process. You can also use release zip files if you wish.

# Fetch Crypto++ sources
git clone https://github.com/weidai11/cryptopp
cd cryptopp
git checkout CRYPTOPP_7_0_0

# Fetch Autotools sources
git clone https://github.com/noloader/cryptopp-autotools
cd cryptopp-autotools
git checkout CRYPTOPP_7_0_0
cp Makefile.am configure.ac libcryptopp.pc.in ../
cd ../

# Configure Autotools
autoupdate && autoreconf -f -i
CXX=/opt/developerstudio12.5/bin/CC ./configure

# Build the library
make -j 4

# Run the self tests in-place
./cryptestcwd v
# Finish with "All tests passed"

./cryptestcwd tv all
# Finish with "0 failures"

# Install the library
sudo make install

If you build the 32-bit version of the library and you experience a crash during ./cryptestcwd v, then you want the patch at Commit 8197732756dd. We hoped Oracle would fix the compiler bug but we think it is a hopeless cause.

$ ./cryptestcwd v
...

BlumBlumShub validation suite running...

passed    49ea2cfdb01064a0bbb92af101dac18a94f7b7ce
passed    2af101dac18a94f7b7ce
passed    744548aeacb70edfafd7d50e8e2983756b2746a1

DH validation suite running...

Segmentation Fault (core dumped)

Testing Autotools

The Autotools project is tested on Noloader's GitHub. The test results are available at NoLoader | Crypto++. The test is performed through a script located in TestScripts. The Autotools test script can be run manually.

The test script downloads the Autotools project files to the cryptopp directory. The script then autoreconf's, consfigure's, builds the library and finally executes the self tests.

cd cryptopp
cp TestScripts/cryptest-autotools .

./cryptest-autotools
...