Heap Corruption - Segv_maperr In Android Native Code

I am trying to create a small library for stream AES encryption, I started my work based on Facebook Conceal project (https://github.com/facebook/conceal), just changing some thing

Solution 1:

It is working and it can decipher files without problems but I get random Heap Memory Corruptions when I work with large streams.

From above line it looks to me that your program is clearly overwriting the memory which was allocated implicitly or explicitly by your code. I was trying to understand your code however it was not clear to me. But I tried to look from memory corruption scenario and found that your program does have malloc/free call which might lead to memory overrun.

EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX*) malloc(sizeof(EVP_CIPHER_CTX));
EVP_CIPHER_CTX_init(ctx);

EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX*) malloc(sizeof(EVP_CIPHER_CTX));
EVP_CIPHER_CTX_init(ctx);

I tried to check the layout of the EVP_CIPHER_CTX structure but it was not available in your code. But I saw that these pointers are getting used in various context within your program. Now you should check that under which scenario your buffer can be overwritten as some places you have used different keyLength and depending on this your program is executing different function. I think you may want to review these codes and see whether overflow is possible!!!....

As your application would be running on android based system where we can not run any dynamic tool(Valgrind/WinDBG/Pageheap..) so I guess you need to review your code by putting some log at important place and see where you are overwriting.

Hope above information would be useful for you to understand your problem.

Solution 2:

After all I was able to work around this problem, EVP_CipherUpdate (or jni ReleaseByteArrayElements) sometimes overflow the output buffer causing the heap corruption, nothing in my code was wrong and also it was not a problem with the caller as replacing EVP_CipherUpdate with a memcpy call with the same parameters worked as expected and there was no heap corruption.

So the solution was adding some extra length to the output buffer sent to nativeUpdate and the error was gone.

I have made the full working version of the library for others to use at: https://github.com/frisco82/conceal