C++ Reference

Global Settings

class Settings

Provides access to global settings and version utility methods

Public Static Functions

static void treatIntegersAsDoubles(bool flag)

Sets treatment of integers in ODB-2 data as doubles

Parameters:

flag – Whether to treat integers as doubles (true) or longs (false)

static long integerMissingValue()

Returns the value that identifies a missing integer

Returns:

Missing integer value

static void setIntegerMissingValue(long val)

Sets the value that identifies a missing integer

Parameters:

val – Missing integer value

static double doubleMissingValue()

Returns the value that identifies a missing double

Returns:

Missing double value

static void setDoubleMissingValue(double val)

Sets the value that identifies a missing double

Parameters:

val – Missing double value

static const std::string &version()

Returns release version of the library in human-readable format, e.g. 1.3.0

Returns:

Release version

static const std::string &gitsha1()

Returns version control checksum of the latest change, e.g. a88011c007a0db48a5d16e296934a197eac2050a

Returns:

Version control checksum

Values and Types

enum odc::api::ColumnType

Identifies the type of data encoded in a column

Values:

enumerator IGNORE

Specifies that the column is ignored (invalid for real data)

enumerator INTEGER

Specifies the column contains integer data

enumerator REAL

Specifies the column contains 32-bit floating point values

enumerator STRING

Specifies the column contains character (string) data

enumerator BITFIELD

Specifies the column contains bitfield data

enumerator DOUBLE

Specifies the column contains 64-bit floating point values

struct ColumnInfo

Describes a column and the data encoded in it

Public Members

std::string name = {}

Column name

ColumnType type = {}

Column data type

size_t decodedSize = {}

Size of a single decoded value in bytes

std::vector<Bit> bitfield = {}

List of bit and bit groups associated with a bitfield column

struct Bit

Describes a bit or bit group in a bitfield

Public Members

std::string name

Bit group name

int size

Bit group size in bits

int offset

Bit group offset in bits

typedef StridedDataT<char> odc::api::StridedData

Describes the layout of periodic data in memory (mutable)

typedef StridedDataT<const char> odc::api::ConstStridedData

Describes the layout of periodic data in memory (const)

template<typename value_type>
class StridedDataT

Describes the layout of periodic data in memory. This is the template base class for the StridedData and ConstStridedData types.

Public Functions

inline StridedDataT(void_arg_t data, size_t nelem, size_t dataSize, size_t stride)

Constructor

Parameters:

  • data – Data array

  • nelem – Number of data elements

  • dataSize – Size of each data element

  • stride – Size between periodic data elements

inline StridedDataT<value_type> slice(size_t rowOffset, size_t nrows)

Returns a new object which references a contiguous subset of the data elements of the original

Parameters:

  • rowOffset – Data element offset where to start slicing

  • nrows – Number of data elements to slice

Returns:

Subset of current data

inline size_t nelem() const

Returns number of data elements

Returns:

Number of data elements

inline size_t dataSize() const

Returns size of each data element

Returns:

Size of each data element

inline size_t stride() const

Returns size between periodic data elements

Returns:

Size between periodic data elements

inline value_type *get(int i)

Returns the address of the i’th data element (mutable)

Parameters:

i – Periodic offset

Returns:

Target mutable data element

inline const_value_type *get(int i) const

Returns the address of the i’th data element (const)

Parameters:

i – Periodic offset

Returns:

Target constant data element

inline void fill(int sourceRow, int finalRow)

Copy the value contained in one data element into the following contiguous elements.

Parameters:

  • sourceRow – Source data element offset

  • finalRow – Target data element offset

inline bool isNewValue(size_t row) const

Checks if data element differs from the previous data element

Parameters:

  • row – Data element offset to check

  • <em>True</em> – if data element does not exist yet, false otherwise

Reader API

class Reader

Owns the ODB-2 data stream and controls associated resources. The Reader object gives access to the sequence of frames.

Public Functions

Reader(const std::string &path, bool aggregated = true, long rowlimit = -1)

Construct from file path

Parameters:

  • path – File path to open

  • aggregated – Whether to aggregate compatible data into a logical frame

  • rowlimit – Maximum number of rows to aggregate into one logical frame

Reader(eckit::DataHandle &dh, bool aggregated = true, long rowlimit = -1)

Construct from data handle reference. This does not take ownership of the data handle, and managing the lifetime of this data handle is the responsibility of the caller.

Parameters:

  • dh – Data handle (eckit)

  • aggregated – Whether to aggregate compatible data into a logical frame

  • rowlimit – Maximum number of rows to aggregate into one logical frame

Reader(eckit::DataHandle *dh, bool aggregated = true, long rowlimit = -1)

Construct via data handle pointer. This takes ownership of the DataHandle.

Parameters:

  • dh – Data handle (eckit)

  • aggregated – Whether to aggregate compatible data into a logical frame

  • rowlimit – Maximum number of rows to aggregate into one logical frame

Frame next()

Advances to the next frame in the stream

Returns:

Next frame

Frame Handling

class Frame

Provides a viewport onto a chunk of contiguous, compatible data within the ODB-2 stream. This may be a logical Frame comprising multiple underlying frames in the data stream.

Public Functions

size_t rowCount() const

Returns number of rows in current frame

Returns:

Number of rows

size_t columnCount() const

Returns number of columns in current frame

Returns:

Number of columns

eckit::Offset offset() const

Returns current frame data offset within the data stream owned by the Reader

Returns:

Frame offset

eckit::Length length() const

Returns the size of the encoded data in bytes

Returns:

Frame length

const std::vector<ColumnInfo> &columnInfo() const

Returns current frame column information

Returns:

Column information

bool hasColumn(const std::string &name) const

Checks if frame has a named column

Parameters:

name – Column name

Returns:

True if named column exists, false otherwise

Frame filter(const std::string &sql)

Filters current frame according to an SQL-like query and returns another frame object (which owns its own attached memory buffer) buffer

Parameters:

sql – SQL query

Returns:

Frame object attached to a memory buffer

eckit::Buffer encodedData()

Returns the encoded data of the current frame

Returns:

Encoded frame data

Span span(const std::vector<std::string> &columns, bool onlyConstantValues) const

Returns a Span object describing the range of values in the Frame associated with the specified columns, without decoding the frame.

Parameters:

  • columns – List of column names

  • onlyConstantValues – Introduces a constraint that a frame must have constant values in a column if true

Returns:

Span object

const std::map<std::string, std::string> &properties() const

Returns key/value properties encoded in the current frame

Returns:

Map object

class Span

Provides details of the range of values spanned by each column within a given frame

Public Functions

void visit(SpanVisitor &visitor) const

Visit the Span object with the specified visitor. Calls the appropriate typed method on the visitor for each column in the Span with the set of associated values.

Parameters:

visitor – Span visitor instance

const std::set<long> &getIntegerValues(const std::string &column) const

Returns integer values present in the specified column

Parameters:

column – Column name

Returns:

Integer values

const std::set<double> &getRealValues(const std::string &column) const

Returns floating point values present in the specified column

Parameters:

column – Column name

Returns:

Real values

const std::set<std::string> &getStringValues(const std::string &column) const

Returns string values present in the specified column

Parameters:

column – Column name

Returns:

String values

bool operator==(const Span &rhs) const

Compare Spans for equality

Parameters:

rhs – Reference to span object to compare with

Returns:

Returns true if span objects are equal

Span &operator=(Span &&rhs)

Move assignment operator

Parameters:

rhs – Reference to a span value to assign

eckit::Offset offset() const

Returns the offset of the associated frame in the data stream

Returns:

Span offset

eckit::Length length() const

Returns the length of the encoded data of the associated frame in the data stream

Returns:

Span length

class SpanVisitor

Provides an interface for accessing the contents of a Span class using a visitor pattern. The appropriate operator() will be called for each column with its associated values.

Public Functions

virtual void operator()(const std::string &columnName, const std::set<long> &vals) = 0

Method for processing long values

Parameters:

  • columnName – Column name

  • vals – Long values

virtual void operator()(const std::string &columnName, const std::set<double> &vals) = 0

Method for processing double values

Parameters:

  • columnName – Column name

  • vals – Double values

virtual void operator()(const std::string &columnName, const std::set<std::string> &vals) = 0

Method for processing string values

Parameters:

  • columnName – Column name

  • vals – String values

Decoding Data

class Decoder

Specifies the ODB-2 columns to decode and the memory layout for the decoded data

Public Functions

Decoder(const std::vector<std::string> &columns, std::vector<StridedData> &columnFacades)

Constructor

Parameters:

  • columns – The names of the columns to decode

  • columnFacades – A description of the periodic data layout for each named column

Decoder slice(size_t rowOffset, size_t nrows) const

Obtain a sub-decoder associated with a contiguous subset of the rows reference by the main decoder

Parameters:

  • rowOffset – Row offset where to start slicing

  • nrows – Number of rows to slice

Returns:

Sliced decoder

void decode(const Frame &frame, size_t nthreads = 1)

Decodes passed frame according to current configuration

Parameters:

  • frame – Frame object

  • nthreads – Number of threads

Encoding Data

void odc::api::encode(eckit::DataHandle &out, const std::vector<ColumnInfo> &columns, const std::vector<ConstStridedData> &data, const std::map<std::string, std::string> &properties = {}, size_t maxRowsPerFrame = 10000)

Encodes ODB-2 into a data handle

Parameters:

  • out – Data handle (eckit)

  • columns – Name and type specification for each column to encode

  • data – Description of the periodic data layout for each column to encode

  • properties – Dictionary of key/value properties to encode

  • maxRowsPerFrame – Maximum number of rows per frame

Conversion Functions

size_t odc::api::filter(const std::string &sql, eckit::DataHandle &in, eckit::DataHandle &out)

Filters ODB-2 data according to an SQL-like query and writes result into another data handle

Note

Depending on the query, SQL filtering may not be appropriate to do on a per-Frame basis, as aggregate values won’t behave properly. This function allows filtering of an entire data stream.

Parameters:

  • sql – SQL query

  • in – Source data handle

  • out – Target data handle

Returns:

Number of rows written

size_t odc::api::odbFromCSV(const std::string &in, eckit::DataHandle &dh_out, const std::string &delimiter = ",")

Imports CSV from a string into a ODB-2 data handle

Parameters:

  • is – String containing CSV data

  • dh_out – ODB-2 data handle (eckit)

  • delimiter – CSV delimiter

Returns:

Number of imported lines

size_t odc::api::odbFromCSV(eckit::DataHandle &dh_in, eckit::DataHandle &dh_out, const std::string &delimiter = ",")

Imports CSV from a source data handle into a ODB-2 data handle

Parameters:

  • dh_in – Input data stream (CSV data)

  • dh_out – ODB-2 data handle (eckit)

  • delimiter – CSV delimiter

Returns:

Number of imported lines

size_t odc::api::odbFromCSV(std::istream &is, eckit::DataHandle &dh_out, const std::string &delimiter = ",")

Imports CSV from a stream handle into a ODB-2 data handle

Parameters:

  • is – Input data stream (CSV data)

  • dh_out – ODB-2 data handle (eckit)

  • delimiter – CSV delimiter

Returns:

Number of imported lines