Question

I’m pretty sure this will end up being a really obvious question, and that’s why I haven’t found much information on it. Still, I thought it was worth asking 🙂

Basically, accessing data using a struct is really fast. If data comes off the network in a form where it can be immediately processed as a struct, this is pretty sweet from a performance point of view.

However, is it possible to define a struct dynamically. Could a client and server app negotiate the format of the datastream and then use that definition as a struct?

If not, is there a better way of doing it?

Thanks all!

Answer 1

It isn’t possible to dynamically define a struct that is identical to a compile-time struct.

It is possible, but difficult, to create dynamic structures that can contain the information equivalent to a struct. The access to the data is less convenient than what is available at compile-time.

All else apart, you cannot access a member somestruct.not_seen_at_compile_time using the dot . or arrow -> notation if it was not defined at compile-time.

With network communications, there are other issues to address – notably ‘endianness’. That is, the data on the wire will probably include multi-byte (2, 4, 8) integers, and either the MSB or the LSB will be sent first, but if one machine is little-endian (IA-32, IA-64, x86/64) and the other is big-endian (SPARC, PPC, almost anything not from Intel), then the data will need to be transformed. Floating-point formats can also be problematic. There are numerous standards dedicated to defining how data will be sent across the network – it is not trivial. Some are specific: IP, TCP, UDP; others are general, such as ASN.1.

However, the ‘cannot do dynamic data structures’ part limits things – you have to agree beforehand on what the data structures are, and how they will be interpreted.

---

How do you do that?

gerty3000 asks:

It is possible, but difficult, to create dynamic structures that can contain the information equivalent to a struct. — How do you do that? I would like to pass dynamically-defined structs off to other C code (assume same compiler and other settings) without having to duplicate the struct memory layout routines from the compiler. I won’t be accessing fields of these structs inside my process much (just initializing them once), so convenient syntax is not a concern.

You can’t do it without duplicating the memory layout in some shape or form. It might not have to be exactly the same, but it is likely best if it is. Here’s some sample code that shows roughly how it might be done.

dynstruct.c

This contains the basic structure manipulation material — structures to describe structures and (simple) members. Handling full arrays (as opposed to strings) would require more work, and there’s a good deal of make-work replication to be managed for other types.

It also contains a main() program that tests the code. It makes a call to other_function(), which demonstrates that the structure I’ve defined in the data structures does match the structure exactly. The data does assume a 64-bit machine where double must be aligned on an 8-byte boundary (so there’s a 4-byte hole in the structure); you will have to tweak the data for a machine where double can be on a 4-byte boundary.

#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

/* This is the type that will be simulated dynamically */
/*
struct simulated
{
    int     number;
    double  value;
    char    string[32];
};
*/

/* SOF structure.h */
typedef enum Type { INT, DOUBLE, STRING } Type;

typedef struct Descriptor
{
    size_t  offset;
    Type    type;
    size_t  type_size;
    size_t  array_dim;
    char    name[32];
} Descriptor;

typedef struct Structure
{
    size_t      size;
    char        name[32];
    Descriptor *details;
} Structure;

extern void   *allocate_structure(const Structure *structure);
extern void    deallocate_structure(void *structure);
extern void   *pointer_to_element(void *p, const Descriptor *d);
extern int     get_int_element(void *p, const Descriptor *d);
extern void    set_int_element(void *p, const Descriptor *d, int newval);
extern double  get_double_element(void *p, const Descriptor *d);
extern void    set_double_element(void *p, const Descriptor *d, double newval);
extern char   *get_string_element(void *p, const Descriptor *d);
extern void    set_string_element(void *p, const Descriptor *d, char *newval);
/* EOF structure.h */

static Descriptor details[] =
{
    {   0,  INT,    sizeof(int),     1, "number"    },
    {   8,  DOUBLE, sizeof(double),  1, "value"     },
    {  16,  STRING, sizeof(char),   32, "string"    },
};

static Structure simulated = { 48, "simulated", details };

void *allocate_structure(const Structure *structure)
{
    void *p = calloc(1, structure->size);
    return p;
}

void deallocate_structure(void *structure)
{
    free(structure);
}

void *pointer_to_element(void *p, const Descriptor *d)
{
    void *data = (char *)p + d->offset;
    return data;
}

int get_int_element(void *p, const Descriptor *d)
{
    assert(d->type == INT);
    int *v = pointer_to_element(p, d);
    return *v;
}

void set_int_element(void *p, const Descriptor *d, int newval)
{
    assert(d->type == INT);
    int *v = pointer_to_element(p, d);
    *v = newval;
}

double get_double_element(void *p, const Descriptor *d)
{
    assert(d->type == DOUBLE);
    double *v = pointer_to_element(p, d);
    return *v;
}

void set_double_element(void *p, const Descriptor *d, double newval)
{
    assert(d->type == DOUBLE);
    double *v = pointer_to_element(p, d);
    *v = newval;
}

char *get_string_element(void *p, const Descriptor *d)
{
    assert(d->type == STRING);
    char *v = pointer_to_element(p, d);
    return v;
}

void set_string_element(void *p, const Descriptor *d, char *newval)
{
    assert(d->type == STRING);
    assert(d->array_dim > 1);
    size_t len = strlen(newval);
    if (len > d->array_dim)
        len = d->array_dim - 1;
    char *v = pointer_to_element(p, d);
    memmove(v, newval, len);
    v[len] = '\0';
}

extern void other_function(void *p);

int main(void)
{
    void *sp = allocate_structure(&simulated);

    if (sp != 0)
    {
        set_int_element(sp, &simulated.details[0], 37);
        set_double_element(sp, &simulated.details[1], 3.14159);
        set_string_element(sp, &simulated.details[2], "Absolute nonsense");
        printf("Main (before):\n");
        printf("Integer: %d\n", get_int_element(sp, &simulated.details[0]));
        printf("Double:  %f\n", get_double_element(sp, &simulated.details[1]));
        printf("String:  %s\n", get_string_element(sp, &simulated.details[2]));
        other_function(sp);
        printf("Main (after):\n");
        printf("Integer: %d\n", get_int_element(sp, &simulated.details[0]));
        printf("Double:  %f\n", get_double_element(sp, &simulated.details[1]));
        printf("String:  %s\n", get_string_element(sp, &simulated.details[2]));

        deallocate_structure(sp);
    }
    return 0;
}

other.c

This code knows nothing about the structure description material in dynstruct.c; it knows about the struct simulated that the simulation code simulates. It prints the data it is passed and modifies it.

#include <stdio.h>
#include <string.h>

extern void other_function(void *p);

struct simulated
{
    int     number;
    double  value;
    char    string[32];
};

void other_function(void *p)
{
    struct simulated *s = (struct simulated *)p;

    printf("Other function:\n");
    printf("Integer: %d\n", s->number);
    printf("Double:  %f\n", s->value);
    printf("String:  %s\n", s->string);

    s->number *= 2;
    s->value  /= 2;
    strcpy(s->string, "Codswallop");
}

Sample output

Main (before):
Integer: 37
Double:  3.141590
String:  Absolute nonsense
Other function:
Integer: 37
Double:  3.141590
String:  Absolute nonsense
Main (after):
Integer: 74
Double:  1.570795
String:  Codswallop

Clearly, this code is not production-ready. It is a sufficient demonstration of what can be done. One issue you’d have to deal with is initializing the Structure and Descriptor data correctly. You can’t put too many assertions into that sort of code. For example, I should really have assert(d->size == sizeof(double); in get_double_element(). It would also be sensible to include assert(d->offset % sizeof(double) == 0); to ensure that the double element is properly aligned. Or you might have a validate_structure(const Structure *sp); function that did all these validation checks. You’d need a function void dump_structure(FILE *fp, const char *tag, const Structure *sp); to dump the defined structure to the given file preceded by the tag, to assist in debugging. Etc.

This code is pure C; it is not compilable by a C++ compiler as C++. There aren’t enough casts to satisfy a C++ compiler.