HDF5 (baseband_tasks.io.hdf5)

hdf5 contains interfaces for reading and writing an internal HDF5 format that is suitable for storing intermediate steps of pipelines. A writer can conveniently be used as the output argument for any reader, automatically writing in the relevant block size. For instance:

.. doctest-requires:: h5py, pyyaml
>>> from baseband import data, vdif
>>> from baseband_tasks.functions import Square
>>> from baseband_tasks.io import hdf5
>>> fh = vdif.open(data.SAMPLE_VDIF, 'rs')
>>> square = Square(fh)
>>> squared = square.read()
>>> h5w = hdf5.open('squared.hdf5', 'w', template=square)
>>> square.seek(0)
0
>>> square.read(out=h5w)
<HDF5StreamWriter name=squared.hdf5 offset=40000
     sample_rate=32.0 MHz, samples_per_frame=40000,
     sample_shape=(8,), dtype=float32,
     start_time=2014-06-16T05:56:07.000000000>
>>> h5w.close()
>>> fh.close()
>>> h5r = hdf5.open('squared.hdf5', 'r')
>>> recovered = h5r.read()
>>> (squared == recovered).all()
True

It is also possible to stored data encoding using the standard vdif schemes by passing in bps and complex_data. Alternatively, half-precision floats can be used by passing in encoded_dtype='f2' or 'c4'.

Reference/API

baseband_tasks.io.hdf5 Package

Functions

open(filename[, mode])

Open an HDF5 file as a stream.

Classes

HDF5Frame(header, payload[, valid, verify])

Representation of a HDF5 frame, consisting of a header and payload.

HDF5Header(*[, verify, mutable])

HDF5 format header.

HDF5Payload(words[, header])

Container for decoding and encoding HDF5 payloads.

Class Inheritance Diagram

Inheritance diagram of baseband_tasks.io.hdf5.frame.HDF5Frame, baseband_tasks.io.hdf5.header.HDF5Header, baseband_tasks.io.hdf5.payload.HDF5Payload

baseband_tasks.io.hdf5.header Module

Definitions for HDF5 general storage headers.

Implements a HDF5Header class used to store header definitions, and provides methods to initialize from a stream template, and to write to and read from an HDF5 Dataset, encoded as yaml file.

Classes

HDF5Header(*[, verify, mutable])

HDF5 format header.

HDF5RawHeader(*[, verify, mutable])

HDF5CodedHeader(*[, verify, mutable])

Class Inheritance Diagram

Inheritance diagram of baseband_tasks.io.hdf5.header.HDF5Header, baseband_tasks.io.hdf5.header.HDF5RawHeader, baseband_tasks.io.hdf5.header.HDF5CodedHeader

baseband_tasks.io.hdf5.payload Module

Payload for HDF5 format.

Classes

HDF5Payload(words[, header])

Container for decoding and encoding HDF5 payloads.

HDF5RawPayload(words[, header])

HDF5CodedPayload(words[, header])

HDF5DatasetWrapper(words)

Make a HDF5 Dataset look a bit more like ndarray.

Variables

DTYPE_C4

Numpy dtype used to encode half-precision complex numbers.

Class Inheritance Diagram

Inheritance diagram of baseband_tasks.io.hdf5.payload.HDF5Payload, baseband_tasks.io.hdf5.payload.HDF5RawPayload, baseband_tasks.io.hdf5.payload.HDF5CodedPayload, baseband_tasks.io.hdf5.payload.HDF5DatasetWrapper

baseband_tasks.io.hdf5.base Module

Interfaces for reading and writing from an internal HDF5 format.

In this format, each HDF5 File has ‘header’ and ‘payload’ h5py.Dataset instances, with the header consisting of yaml-encoded keywords describing the start time, sample rate, etc., and the payload consisting of either plain numpy data, or data encoded following the VDIF standard.

Functions

open(filename[, mode])

Open an HDF5 file as a stream.

Classes

HDF5StreamBase(fh_raw, header0, **kwargs)

HDF5StreamReader(fh_raw[, squeeze, subset, ...])

HDF5StreamWriter(fh_raw[, header0, template])

Class Inheritance Diagram

Inheritance diagram of baseband_tasks.io.hdf5.base.HDF5StreamBase, baseband_tasks.io.hdf5.base.HDF5StreamReader, baseband_tasks.io.hdf5.base.HDF5StreamWriter