Make sure we null-terminate encrypted strings before passing them to
UTF8ToString.
This used to work when we allocated the buffer on the stack, because it turns
out that allocate() zeroinits the returned memory. malloc(), of course, does
not.
Messages can be very large, so we don't really want to allocate them on the
stack. Switch to using the heap for them, and try to clean up some of the
string handling while we're at it.
Repeat the fix from b10f90d for megolm messages.
It turns out that the 'length' argument to 'Pointer_stringify' doesn't work if
the input includes characters >= 128.
Rather than try to figure out which methods can return UTF-8, and which always
return plain ascii, replace all uses of Pointer_stringify with a 'length'
argument with the version that expects a NULL-terminated input, and extend the
buffer by a byte to allow space for a null-terminator.
In the case of decrypt, we need to add the null ourself.
Fixes https://github.com/vector-im/vector-web/issues/2078.
Ed25519 private keys, it turns out, have 64 bytes, not 32.
We were previously generating only 32 bytes (which is all that is required to
generate the public key), and then using the public key as the upper 32 bytes
when generating the per-message session key. This meant that everything
appeared to work, but the security of the private key was severely compromised.
By way of fixes:
* Use the correct algorithm for generating the Ed25519 private key, and store
all 512 bits of it.
* Update the account pickle format and refuse to load the old format (since we
should consider it compromised).
* Bump the library version, and add a function to retrieve the library
version, so that applications can verify that they are linked against a
fixed version of the library.
* Remove the curve25519_{sign, verify} functions which were unused and of
dubious quality.
It turns out that the 'length' argument to 'Pointer_stringify' doesn't work if
the input includes characters >= 128.
Rather than try to figure out which methods can return UTF-8, and which always
return plain ascii, replace all uses of Pointer_stringify with a 'length'
argument with the version that expects a NULL-terminated input, and extend the
buffer by a byte to allow space for a null-terminator.
In the case of decrypt, we need to add the null ourself.
Fixes https://github.com/vector-im/vector-web/issues/1719.
