Random Musings of an Insane Mind

Ages ago we had a system that used MySQL’s built in DES3 encryption. It made coding applications in multiple languages easy because we could send it a string with a key and it would be encoded in the database. It wasn’t secure if someone got hold of the code and the database, but, rather than use a hash, we could store things and get the plaintext back if we needed it. Due to some policy issues with Debian, DES3 encryption was removed from MySQL and we were faced with converting that data to another format. We chose RSA for the fact that it was supported in every language we were currently developing in — Perl, PHP and C, and we knew it was supported in Python even though we hadn’t started development with Python at the time.

However, due to issues with PHP and long key lengths (at the time the code was written), our payloads had to be broken into packets smaller than the smallest key length which was 256 bytes. Since we didn’t know if we would run into similar issues using a longer packet length even if we had a longer key, we opted to use a packet size smaller than the smallest key we could generate. Our code initially converted data using PHP, stored the data in MySQL, and allowed a Perl script to access the data. Getting Perl and PHP to cooperate was somewhat difficult until we dug into PHP’s source code to see just how they were handling RSA.

PHP code:

define("ENCPAYLOAD_FORMAT",'Na*');

function cp_encrypt($hostname,$message) {
  $public_key=file_get_contents(KEY_LOCATION . $hostname . '.public.key');
  openssl_get_publickey($public_key);

  $blockct = intval(strlen($message) / 245)  + 1;
  $encpayload = "";
  for ($loop=0;$loop<$blockct;$loop++) {
    $blocktext = substr($message,$loop * 245, 245);
    openssl_public_encrypt($blocktext,$encblocktext,$public_key);
    $encpayload .= $encblocktext;
  }
  return(pack(ENCPAYLOAD_FORMAT,$blockct,$encpayload));
}

function cp_decrypt($hostname,$message) {
  $priv_key=file_get_contents(KEY_LOCATION . $hostname . '.private.key');
  openssl_get_privatekey ($priv_key);
  $arr = unpack('Nblockct/a*',$message);
  $blockct = $arr['blockct'];$encpayload=$arr[1];
  $decmessage = "";
  for ($loop=0;$loop<$blockct;$loop++) {
    $blocktext = substr($encpayload, $loop*256, 256);
    openssl_private_decrypt($blocktext,$decblocktext,$priv_key);
    $finaltext .= $decblocktext;
  }

  return($finaltext);
}

Perl Code:

use Crypt::OpenSSL::RSA;
use constant ENCPAYLOAD_FORMAT => 'Na*';

sub cp_encrypt {
  my $hostname = shift;
  my $message = shift;

  my $keyfile = $KEY_LOCATION . $hostname . '.public.key';
  
  if (-e $keyfile) {
    open PUBLIC, $keyfile;
    my $public_key = do{local $/; };
    close(PUBLIC);
    my $rsa = Crypt::OpenSSL::RSA->new_public_key($public_key);
    $rsa->use_pkcs1_padding();

    my $blockct = int(length($message) / 245)  + 1;
    my $encpayload = "";
      for ($loop=0;$loop<$blockct;$loop++) {
        $encpayload .= $rsa->encrypt(substr($message,$loop * 245, 245));
      }
    return(pack(ENCPAYLOAD_FORMAT,$blockct,$encpayload));
  }
  return(-1);
}

sub cp_decrypt {
  my $hostname = shift;
  my $message = shift;

  my $keyfile = $KEY_LOCATION . $hostname . '.private.key';
  if (-e $keyfile) {
    open PRIVATE, $keyfile;
    my $private_key = do{local $/; };
    close(PRIVATE);
    my $rsa = Crypt::OpenSSL::RSA->new_private_key($private_key);
    $rsa->use_pkcs1_padding();
    my ($blockct,$encpayload) = unpack(ENCPAYLOAD_FORMAT,$message);
    my $decmessage = "";
    for ($loop=0;$loop<$blockct;$loop++) {
      $decmessage .= $rsa->decrypt(substr($encpayload, $loop*256, 256));
    }
    return($decmessage);
  }
  return(-1);
}

1;

Python code:

import M2Crypto.RSA
from os import path
from struct import unpack, pack, calcsize

ENCPAYLOAD_FORMAT = 'Na*'

def perl_unpack (perlpack, payload):
    if (perlpack == 'Na*'):
        count = calcsize('!L')
        perlpack = '!L%ds' % (len(payload) - count)
        return unpack(perlpack,payload)
    return

def perl_pack (perlpack, blockcount, payload):
    if (perlpack == 'Na*'):
        perlpack = '!L%ds' % len(payload)
        return pack(perlpack,blockcount,payload)
    return

def cp_encrypt (hostname,message):
  keyfile = KEY_LOCATION + hostname + '.public.key'
  if (path.exists(keyfile)):
    public_key = M2Crypto.RSA.load_pub_key(keyfile)

    blockct = int(len(message) / 245)  + 1
    encpayload = ""
    for loop in range(0,blockct):
      encpayload += public_key.public_encrypt(message[(loop*245):(245*(loop+1))],
                    M2Crypto.RSA.pkcs1_padding)
    return(perl_pack(ENCPAYLOAD_FORMAT, blockct, encpayload))
  return(-1)

def cp_decrypt (hostname, message):
  keyfile = KEY_LOCATION + hostname + '.private.key';
  if (path.exists(keyfile)):
    privatekey = M2Crypto.RSA.load_key(keyfile)
    (blockct,encpayload) = perl_unpack(ENCPAYLOAD_FORMAT,message)

    decmessage = ""
    for loop in range(0,blockct):
      decmessage += privatekey.private_decrypt(encpayload[(loop*256):(256*(loop+1))], M2Crypto.RSA.pkcs1_padding)
    return(decmessage);
  return(-1)

There is nothing really that restricts you from encrypting a message longer than the key size, but, if the message contains enough data, portions of the key can be discovered. Since the key is a very large prime number, exposing a portion of that key can be vital to decrypting the data.

Perl has very powerful capabilities for dealing with structures.  PHP’s support of those structures was based on Perl’s wisdom.  Python went a different direction.

Perl pack/unpack definitions

PING_FORMAT => ‘(a4n2N2N/a*)@245’;
TASK_FORMAT => ‘a4NIN/a*a*’;
RETR_FORMAT => ‘a4N/a*N’;
ENCPAYLOAD_FORMAT => ‘Na*’;

PHP pack/unpack definitions

define(‘TASK_FORMAT’, ‘a4NINa*a*’);
define(“ENCPAYLOAD_FORMAT”,’Na*’);

For a communications package written in perl that communicates with 32 bit and 64 bit machines that may not share the same endian structure.  The problem I’ve run into now is that Python does not support the Perl method, and, I don’t know why they didn’t at least offer some compatibility.  pack and unpack give enormous power to communication systems between machines and their support of the perl methods allowed for reasonable interoperability between the two platforms.

Python on the other hand opted to not support some of the features, which was one issue, but, their requirement is that you cannot send variable length packets.

In Python, we’re able to replicate N, network endian Long by using !L:

>>> import struct
>>> print struct.unpack(‘!L’,’\0\0\1\0′);
(256,)

However, there is no method to support a variable length payload behind that value.  We’re able to set a fixed length like 5s, but, this means that we’ve got to know the length of the payload being sent.

>>> print struct.unpack(‘!L5s’,’\0\0\1\0abcde’);
(256, ‘abcde’)

If we overstate the size of the field, Python is more than happy to tell us that the payload length doesn’t match the length of the data.

>>> print struct.unpack(‘!L8s’,’\0\0\1\0abcde’);
Traceback (most recent call last):
File “<stdin>”, line 1, in <module>
File “/usr/lib/python2.5/struct.py”, line 87, in unpack
return o.unpack(s)
struct.error: unpack requires a string argument of length 12

The cheeseshop/pypi seems to show no suitable alternative which brings up a quandry.  For this particular solution, I’ll write a wrapper function to do the heavy lifting on the two unpack strings I need to deal with and then I’ll debate pulling the perl unpack/pack routines out of the perl source and wrapping it into an .egg, possibly for distribution.