[vox-tech] Memory addressing?

[Bill Broadley]

On 06/23/2010 10:42 AM, timriley at appahost.com wrote:
> The reason for the discussion was Brian's intrusion detection
> implementation stored the incoming packets in a hash
> table. The key to the hash table was quite
> large -- inbound IP address, outbound IP address, inbound port, and
> outbound port. I thought to my self, on a very large implementation (say
> Google) the table could grow to a billion entries. Could a hash table
> store this amount in memory? Could you allocate an array of half the
> total memory? Could you allocate an array of a billion integers? Brian,
> on his laptop, couldn't allocate a billion integers. But he could
> allocate a billion characters (bytes). Since I thought both bytes and
> integers were words, and since I thought memory stored words
> like registers stored words, we had our discussion.

Sounds like an interesting discussion, sorry I missed it.  Kinda of 
amusing trying to handle such a hash table on a (older I assume) single 
32 bit laptop.

> The following failed to compile with an array-to-large error:
> int main( void )
> {
>       int table[ 1000000000 ];
>       return 0;
> }

Keep in mind, that's allocating on the stack, you could easily run out 
even on a 64 bit machine.  Moving the int table up before the int main 
would allocate it directly from memory.

Also I believe that requires contiguous memory, which may well be 
significantly lower than the free memory.

> The following succeeded to compile:
> int main( void )
> {
>       char table[ 1000000000 ];
>       return 0;
> }
>
> This compiles but core dumps:
> #include<stdio.h>
> int main( void )
> {
>       char table[ 1000000000 ];
>       printf( "Hello world!\n" );
> }

Keep in mind that compilers are getting smarter and I've seen problems 
with blind arrays.  Also depending on the OS they may or may not 
allocate memory that you ask for but don't initialized.  I'm not sure 
offhand if C initializes statically allocated memory like that.  So if 
you want to insure that a memory allocation (generically is working) I'd 
use the memory.  The code I used was:
#include <stdio.h>
#include <stdlib.h>

#define big 1000000000
int table[ big ];

int main( void )
{
       int i;
       for (i=0;i<big;i++)
       {
	table[i]=i/2048;
       }
       printf ("%d\n",table[123]);
       return 0;
}