Hardware-Assisted Checking Using Silicon Secured Memory (SSM)
The SPARC M7 processor from Oracle offers Software in Silicon, which enables software to run faster and more reliably. One Software in Silicon feature is Silicon Secured Memory (SSM), previously called Application Data Integrity (ADI), whose circuitry detects common memory access errors that can cause run-time data corruption.
These errors can be caused by errant code or a malicious attack on a server's memory. For example, buffer overflows are known to be a major source of security exploits. Further, in-memory databases increase an application's exposure to such errors due to having critical data in-memory.
Silicon Secured Memory stops memory corruptions in optimized production code by adding version numbers to the application's memory pointers and the memory they point to. If the pointer version number does not match the content version number, the memory access is aborted. Silicon Secured Memory works with applications written in systems-level programming languages such as C or C++, which are more vulnerable to memory corruption caused by software errors.
Oracle Developer Studio 12.6 includes the libdiscoverADI.so library (also referred to as the discover ADI library), which provides updated malloc() library routines that ensure that adjacent data structures are given different version numbers. These version numbers enable the processor's SSM technology to detect memory access errors, like buffer overflows. Memory content version numbers are changed when memory structures are freed to prevent stale pointer accesses. For more information about the errors caught by discover and libdiscoverADI.so, see Errors Caught by the libdiscoverADI Library.
In addition to using SSM in production to detect potential memory corruption issues, , you can use it during application development to ensure that such errors are caught during application testing and certification. Memory corruption bugs are extremely hard to find because applications encounter corrupted data long after the corruption happens. The discover tool and the libdiscoverADI.so library, part of the Oracle Developer Studio developer tool suite, provide you with additional application information that makes locating and fixing the errant code easier.
Using the libdiscoverADI Library to Find Memory Access Errors
The discover ADI library libdiscoverADI.so reports programming errors that result in invalid memory accesses. You can use in it two ways:
-
By preloading the discover ADI library into your application with the LD_PRELOAD_64. environment variable. This method runs all 64-bit binaries in the application in ADI mode, For example, if you normally run an application named server, the command would be as follows:
$ LD_PRELOAD_64=install-dir/lib/compilers/sparcv9/libdiscoverADI.so server
-
By using ADI mode with the discover command with the –i adi option on a specific binary.
% discover -i adi a.out % a.out
The errors are reported in an a.out.html file by default. For more information about discover reports, see Analyzing discover Reports and Output Options.
See Requirements and Limitations of Using libdiscoverADI.
Errors Caught by the libdiscoverADI Library
The libdiscoverADI.so library catches the following errors:
-
Array out of Bounds Access (ABR/ABW)
-
Freed Memory Access (FMR/FMW)
-
Stale Pointer Access (A special type of FMR/FMW)
-
Unallocated Read/Write (UAR/UAW)
-
Double Free Memory (DFM)
For more information about each of these types of errors, see Memory Access Errors and Warnings.
For a full example, see Example of Using discover ADI Mode.
Instrumentation Options for discover ADI Mode
The following options determine the precision and amount of information generated in the discover report when instrumenting with ADI.
- –A [on | off]
-
When this flag is set to on, the discover ADI library reports the location of the error and the error stack trace. This information is sufficient to catch the error, but it is not always sufficient to fix the error. See SUNW_DISCOVER_OPTIONS Environment Variable on how to change runtime behavior for this option.
This flag also generates information about where the offending memory area was allocated and freed. For example, the output might say that an error was an Array out of Bounds Access and where that array was allocated. If set to off, allocations and stack trace are not reported. The default is on.
Note - Even if –A is set to on, it is possible that ABR/ABW might sometimes be reported as FMR/FMW or UAR/UAW, due to one of the following reasons:-
If the buffer overflow access happens at a large offset after the end of the buffer or before the beginning of the buffer.
-
If libdiscoverADI.so hits a resource limit. In this case, discover might be able to keep the allocation stack trace that is needed to determine if the error is a buffer overflow.
-
- –P [on | off]
-
When this flag is set to off, ADI is run in non-precise mode. In non-precise mode, memory write errors are caught a few instructions (source lines) after the exact instruction is executed. To enable Precise mode, set this flag to on, which is the default.
For better runtime performance, you can specify –A off, –P off, or both options can be set to off.
Custom Memory Allocators and the discover ADI Library
Enterprise applications can often manage their own memory and do not use the system (libc) malloc(3C) library. For these cases, the normal usage of libdiscoverADI.so, which interposes on malloc() will not be able to catch memory corruptions. Oracle Developer Studio provides APIs so the enterprise applications can tell libdiscoverADI.so when a memory area is allocated or freed. libdiscoverADI.so manages the SSM versioning, signal handling, and error reporting. These APIs and libdiscoverADI.so only works for memory allocated by mmap(2) or shmget(2).
The following APIs are provided:
- void *adi_mark_malloc(void *, size_t);
-
The allocator passes a pointer to the memory about to be passed and the size to the client requesting memory. If the memory does not have ADI enabled, the library uses a memcntl(2) call. A versioned pointer is returned, which the allocator must pass to the client.
- void adi_mark_free(void *, size_t);
-
The allocator passes a pointer and the size to the memory area to be freed.
- void *adi_unversion(void *);
-
Allocators can use pointer arithmetic to access a client memory's meta data. Client memory is often versioned, but allocator meta data is not. In such circumstances, you need to use this API. The allocator passes any pointer, and gets back an equivalent unversioned pointer.
- void *adi_version(void *);
-
Sometimes allocators lose track of versioned pointers. This API takes in any pointer, and returns an equivalent pointer with the correct version on it. Any dereference with the return value will not cause an ADI SEGV.
- void *adi_clear_version(void *, size_t);
-
If a memory area is being reused for a different purpose, the ADI version needs to be cleared. For example, if a memory area was versioned for use by an allocator client, and is now being used by the allocator for meta-data. This function takes in a versioned or unversioned pointer and size, sets the version for that area to 0, and returns an unversioned pointer.
Your application might manage its own memory allocation and free lists, for example by allocating large chunks of memory and subdividing it in your program. See Using Application Data Integrity and Oracle Solaris Studio to Find and Fix Memory Access Errors (https://community.oracle.com/docs/DOC-912448) for information about how you can use the ADI versioning APIs to catch errors with your managed memory.
Using Custom Memory Allocators
To use these APIs, include the following header file in your sources:
install-dir/lib/compilers/include/cc/discoverADI_API.h
Then, do one of the following:
-
Link your sources with install-dir/lib/compilers/sparcV9/libadiplugin.so
-
Set the environment variable LD_PRELOAD_64 to install-dir/lib/compilers/sparcV9/libadiplugin.so
-
Run the following command: discover -i adi executable
The following is an example of using custom memory allocators.
Example 1 Using Custom Memory AllocatorsThe following is an example header file:
% cat allocator.h #define GRANULARITY 32 void *mymalloc(size_t len); void myfree(void *ptr);
The following is example source code, using custom memory allocators and the libdiscoverADI library:
% cat allocator.c
#include <sys/mman.h>
#include <stdio.h>
#include <stdlib.h>
#include <ucontext.h>
#include <errno.h>
#include <dlfcn.h>
#include <unistd.h>
#include <assert.h>
#include "allocator.h"
#include "discoverADI_API.h"
#pragma init (setup_allocator)
#pragma fini (takedown_allocator)
#define MAX_ALLOCATIONS 1024
#define START_ADDRESS 0x200000000
uint64_t next_available_address = 0;
size_t allocation_table[MAX_ALLOCATIONS/GRANULARITY];
static void setup_allocator() {
// mmap with a specific address
void *addr = mmap((void *) START_ADDRESS, MAX_ALLOCATIONS,
PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0);
if (addr == MAP_FAILED) {
fprintf(stderr, "mmap failed {%d}\n", errno);
exit(1);
}
next_available_address = (uint64_t) addr;
}
static void takedown_allocator() {
if (munmap((void *) START_ADDRESS, MAX_ALLOCATIONS) != 0) {
fprintf(stderr, "munmap failed {%d}\n", errno);
exit(1);
}
}
// Simple malloc
void *mymalloc(size_t size) {
void *vaddr = (void *) next_available_address;
next_available_address += size;
assert(next_available_address < (START_ADDRESS+MAX_ALLOCATIONS));
int index = ((uint64_t)vaddr-START_ADDRESS)/GRANULARITY;
allocation_table[index] = size;
// Tell libdiscoverADI.so about the allocation, get a versioned
// pointer
vaddr = adi_mark_malloc(vaddr, size);
// Return the versioned pointer
return vaddr;
}
// Simple free
void myfree(void *ptr) {
int index = ((uint64_t)ptr-START_ADDRESS)/GRANULARITY;
size_t size = allocation_table[index];
// Tell libdiscoverADI.so about the free().
adi_mark_free(ptr, size);
}
% cat simple.c
#include <stdio.h>
#include <stdlib.h>
#include "allocator.h"
int main() {
// Allocate
char *buf1 = (char *) mymalloc(GRANULARITY*2);
buf1[0] = 'x';
// Read before free
printf("Read buf1[0] before free: [0x%p] %c\n", buf1, buf1[0]);
// Allocate
char *buf2 = (char *) mymalloc(GRANULARITY*2);
// Buf1fer overflow buf1
printf("Read buf1[%d] past end: [0x%p] %c\n", GRANULARITY*2,
buf1+GRANULARITY*2, buf1[GRANULARITY*2]);
// Free
myfree(buf1);
// Read after free
printf("Read buf1[0] after free: [0x%p] %c\n", buf1, buf1[0]);
return 0;
}
To compile and run this example:
% cc –m64 –g –I install-dir/lib/compilers/include/cc allocator.c simple.c install-dir/lib/compilers/sparcv9/libadiplugin.so –o simple % ./simple
The following is the resulting generated HTML report:
Requirements and Limitations of Using libdiscoverADI
You can use ADI mode of discover only with 64-bit applications on a SPARC M7 chip running at least Oracle Solaris 11.2.8 or Oracle Solaris 11.3.
Similar to instrumenting for memory checking, preloaded libraries might conflict if functions of libdiscoverADI.so interpose on the same allocation functions. See Binaries That Use Preloading or Auditing Are Incompatible for more information.
Other limitations for checking your code with libdiscoverADI include the following:
-
Only heap-checking is available. There is no stack checking, no static array-out-of-bounds checking, and no leak detection.
-
Does not work with applications which use the unused bits in 64-bit addresses for storing meta data. Some 64-bit applications might use the currently unused high bits in 64-bit addresses for storing meta-data, for instance, locks. Such applications will not work with discover in ADI mode because the feature works by using the 4 highest bits in the 64-bit address to store version information.
-
Might not work for applications that do pointer arithmetic with assumptions about heap addresses, for example, the distance between two successive allocations.
-
Unlike in memcheck mode (instrumenting with –i memcheck), ADI mode does not catch errors if the application redefines standard memory allocation functions in the executable. If the application redefines the standard memory allocation functions in a library, then ADI mode works.
-
Resolution for buffer overflow is 64 bytes. For allocations that are 64-byte-aligned, libdiscoverADI.so will catch any overflow by 1 byte or more. For allocations that are not aligned at 64 bytes, it might miss the buffer overflow by a few bytes. In general, overflow by 1 to 63 bytes might not be caught depending on the alignment of the allocation and where libdiscoverADI.so places the allocation in the cache line.
-
There is a slight chance that binaries compiled with –xipo=2 might have a memory-optimized code that manipulates addresses in a way that will lead to false positive SSM errors and as a result also lead to performance degradation due to trap handling.
Example of Using discover ADI Mode
This section provides a code sample with Array-out-of-bounds errors, which are then caught and reported by discover using ADI mode.
Assume the following sample code resides in a file named testcode.c.
#include <stdio.h>
#include <stdlib.h>
#define STRSZ 64
int main()
{
char *arr1 = (char*)malloc(sizeof(char)*STRSZ);
char *arr2 = (char*)malloc(sizeof(char)*STRSZ);
// Buffer overflow due to using "<=" instead of "<"
for (int i=0; i <= STRSZ; i++)
arr1[i] = arr2[i]; // ABR/ABW
free(arr1);
free(arr2);
char *arr3 = (char*)malloc(sizeof(char)*STRSZ);
arr3[0] = arr2[0]; // FMR
arr2[0] = arr3[3]; // FMW
arr3[1] = arr1[1]; // Possible stale pointer/FMR
free(arr2); // Double Free
return 0;
}
You would build the test code with the following command:
$ cc testcode.c -g -m64
To execute this sample application with ADI mode, use the following command:
$ discover -w - -i adi -o a.out.adi a.out $ ./a.out.adi
This command generates the following output, in a discover report. For more information about reading and understanding these reports, see Analyzing discover Reports.
ERROR 1 (ABR): reading memory beyond array bounds at address 0x3fffffff7d47e080 {memory: v8}:
main() + 0x48 <a.c:13>
10:
11: // Buffer overflow due to using "<=" instead of "<"
12: for (int i=0; i <= STRSZ; i++)
13:=> arr1[i] = arr2[i]; // ABR/ABW
14:
15: free(arr1);
16: free(arr2);
was allocated at (0x3fffffff7d47e040, 64 bytes):
main() + 0x1c <a.c:9>
6: {
7:
8: char *arr1 = (char*)malloc(sizeof(char)*STRSZ);
9:=> char *arr2 = (char*)malloc(sizeof(char)*STRSZ);
10:
11: // Buffer overflow due to using "<=" instead of "<"
12: for (int i=0; i <= STRSZ; i++)
ERROR 2 (ABW): writing to memory beyond array bounds at address 0x2fffffff7d47e040 {memory: v3}:
main() + 0x54 <a.c:13>
10:
11: // Buffer overflow due to using "<=" instead of "<"
12: for (int i=0; i <= STRSZ; i++)
13:=> arr1[i] = arr2[i]; // ABR/ABW
14:
15: free(arr1);
16: free(arr2);
was allocated at (0x2fffffff7d47e000, 64 bytes):
main() + 0x8 <a.c:8>
5: int main()
6: {
7:
8:=> char *arr1 = (char*)malloc(sizeof(char)*STRSZ);
9: char *arr2 =(char*)malloc(sizeof(char)*STRSZ);
10:
11: // Buffer overflow due to using "<=" instead of "<"
ERROR 3 (FMR): reading from freed memory at address 0x3fffffff7d47e040 {memory: v10}:
main() + 0xa0 <a.c:20>
17:
18: char *arr3 = (char*)malloc(sizeof(char)*STRSZ);
19:
20:=> arr3[0] = arr2[0]; // FMR
21: arr2[0] = arr3[3]; // FMW
22: arr3[1] = arr1[1]; // Possible stale pointer/FMR
23:
was allocated at (0x3fffffff7d47e040, 64 bytes):
main() + 0x1c <a.c:9>
6: {
7:
8: char *arr1 = (char*)malloc(sizeof(char)*STRSZ);
9:=> char *arr2 = (char*)malloc(sizeof(char)*STRSZ);
10:
11: // Buffer overflow due to using "<=" instead of "<"
12: for (int i=0; i <= STRSZ; i++)
freed at (0x3fffffff7d47e040, 64 bytes):
main() + 0x80 <a.c:16>
13: arr1[i] = arr2[i]; // ABR/ABW
14:
15: free(arr1);
16:=> free(arr2);
17:
18: char *arr3 = (char*)malloc(sizeof(char)*STRSZ);
19:
ERROR 4 (FMW): writing to freed memory at address 0x3fffffff7d47e040 {memory: v10}:
main() + 0xb8 <a.c:21>
18: char *arr3 = (char*)malloc(sizeof(char)*STRSZ);
19:
20: arr3[0] = arr2[0]; // FMR
21:=> arr2[0] = arr3[3]; // FMW
22: arr3[1] = arr1[1]; // Possible stalepointer/FMR
23:
24: free(arr2); // Double Free
was allocated at (0x3fffffff7d47e040, 64 bytes):
main() + 0x1c <a.c:9>
6: {
7:
8: char *arr1 = (char*)malloc(sizeof(char)*STRSZ);
9:=> char *arr2 =(char*)malloc(sizeof(char)*STRSZ);
10:
11: // Buffer overflow due to using "<=" instead of "<"
12: for (int i=0; i <= STRSZ; i++)
freed at (0x3fffffff7d47e040, 64 bytes):
main() + 0x80 <a.c:16>
13: arr1[i] = arr2[i]; // ABR/ABW
14:
15: free(arr1);
16:=> free(arr2);
17:
18: char *arr3 = (char*)malloc(sizeof(char)*STRSZ);
19:
ERROR 5 (FMR): reading from freed memory at address 0x2fffffff7d47e001 {memory: v3}:
main() + 0xc0 <a.c:22>
19:
20: arr3[0] = arr2[0]; // FMR
21: arr2[0] = arr3[3]; // FMW
22:=> arr3[1] = arr1[1]; // Possible stale pointer/FMR
23:
24: free(arr2); // Double Free
25:
was allocated at (0x2fffffff7d47e000, 64 bytes):
main() + 0x8 <a.c:8>
5: int main()
6: {
7:
8:=> char *arr1 = (char*)malloc(sizeof(char)*STRSZ);
9: char *arr2 = (char*)malloc(sizeof(char)*STRSZ);
10:
11: // Buffer overflow due to using "<=" instead of "<"
freed at (0x2fffffff7d47e000, 64 bytes):
main() + 0x74 <a.c:15>
12: for (int i=0; i <= STRSZ; i++)
13: arr1[i] = arr2[i]; //ABR/ABW
14:
15:=> free(arr1);
16: free(arr2);
17:
18: char *arr3 = (char*)malloc(sizeof(char)*STRSZ);
ERROR 6 (DFM): double freeing memory at address 0x3fffffff7d47e040 {memory: v10}:
main() + 0xd0 <a.c:24>
21: arr2[0] = arr3[3]; // FMW
22: arr3[1] = arr1[1]; // Possible stale pointer/FMR
23:
24:=> free(arr2); // Double Free
25:
26: return 0;
27: }
was allocated at (0x3fffffff7d47e040, 64 bytes):
main() + 0x1c <a.c:9>
6: {
7:
8: char *arr1 = (char*)malloc(sizeof(char)*STRSZ);
9:=> char *arr2 = (char*)malloc(sizeof(char)*STRSZ);
10:
11: // Buffer overflow due to using "<=" instead of "<"
12: for (int i=0; i <= STRSZ; i++)
freed at (0x3fffffff7d47e040, 64 bytes):
main() + 0x80 <a.c:16>
13: arr1[i] = arr2[i]; // ABR/ABW
14:
15: free(arr1);
16:=> free(arr2);
17:
18: char *arr3 = (char*)malloc(sizeof(char)*STRSZ);
19:
DISCOVER SUMMARY:
unique errors : 6 (6 total)