VDIFFileReader

class baseband.vdif.base.VDIFFileReader(fh_raw)

Bases: baseband.vlbi_base.base.VLBIFileReaderBase

Simple reader for VDIF files.

Wraps a binary filehandle, providing methods to help interpret the data, such as read_frame, read_frameset and get_frame_rate.

Parameters

fh_rawfilehandle

Filehandle of the raw binary data file.

Attributes Summary

fh_raw
info()	Standardized information on file readers.

Methods Summary

close(self)
find_header(self[, pattern, edv, mask, …])	Find the nearest header from the current position.
get_frame_rate(self)	Determine the number of frames per second.
get_thread_ids(self[, check])	Determine the number of threads in the VDIF file.
locate_frames(self, pattern, *[, mask, …])	Use a pattern to locate frame starts near the current position.
read_frame(self[, edv, verify])	Read a single frame (header plus payload).
read_frameset(self[, thread_ids, edv, verify])	Read a single frame (header plus payload).
read_header(self[, edv, verify])	Read a single header from the file.
temporary_offset(self[, offset, whence])	Context manager for temporarily seeking to another file position.

Attributes Documentation

fh_raw = None

info

Standardized information on file readers.

The info descriptor has a number of standard attributes, which are determined from arguments passed in opening the file, from the first header (info.header0) and from possibly scanning the file to determine the duration of frames.

Examples

The most common use is simply to print information:

>>> from baseband.data import SAMPLE_MARK5B
>>> from baseband import mark5b
>>> fh = mark5b.open(SAMPLE_MARK5B, 'rb')
>>> fh.info
File information:
format = mark5b
number_of_frames = 4
frame_rate = 6400.0 Hz
bps = 2
complex_data = False
readable = False

missing:  nchan: needed to determine sample shape, frame rate, ...
          kday, ref_time: needed to infer full times.

>>> fh.close()

>>> fh = mark5b.open(SAMPLE_MARK5B, 'rb', kday=56000, nchan=8)
>>> fh.info
File information:
format = mark5b
number_of_frames = 4
frame_rate = 6400.0 Hz
sample_rate = 32.0 MHz
samples_per_frame = 5000
sample_shape = (8,)
bps = 2
complex_data = False
start_time = 2014-06-13T05:30:01.000000000
readable = True

checks:  decodable: True
>>> fh.close()

Attributes

formatstr or None

File format, or None if the underlying file cannot be parsed.

number_of_framesint

Number of frames in the file.

frame_rateQuantity

Number of data frames per unit of time.

sample_rateQuantity

Complete samples per unit of time.

samples_per_frameint

Number of complete samples in each frame.

sample_shapetuple

Dimensions of each complete sample (e.g., (nchan,)).

bpsint

Number of bits used to encode each elementary sample.

complex_databool

Whether the data are complex.

start_timeTime

Time of the first complete sample.

readablebool

Whether the first sample could be read and decoded.

missingdict

Entries are keyed by names of arguments that should be passed to the file reader to obtain full information. The associated entries explain why these arguments are needed.

checksdict

Checks that were done to determine whether the file was readable (normally the only entry is ‘decodable’).

errorsdict

Any exceptions raised while trying to determine attributes or doing checks. Keyed by the attributes/checks.

warningsdict

Any warnings about the attributes or about the checks. Keyed by the attributes/checks.

Methods Documentation

close(self)

find_header(self, pattern=None, *, edv=None, mask=None, frame_nbytes=None, offset=0, forward=True, maximum=None, check=1)

Find the nearest header from the current position.

Search for a valid header at a given position which is consistent with pattern and/or with a header a frame size ahead. Note that the search is much slower if no pattern is given, as at every position it is tried to read a header, and then check for another one one frame ahead. It helps to pass in edv and frame_nbytes (if known).

If successful, the file pointer is left at the start of the header.

Parameters

patternVDIFHeader, array of byte, or compatible

If given, used for a direct search.

edvint

EDV of the header, used if pattern is not given.

maskarray of byte, bytes, iterable of int, string or int

Bit mask for the pattern, with 1 indicating a given bit will be used the comparison. Only used with pattern and not needed if pattern is a header.

frame_nbytesint, optional

Frame size in bytes. Defaults to the frame size in any header passed in.

offsetint, optional

Offset from the frame start that the pattern occurs. Any offsets inferred from masked entries are added to this (hence, no offset needed when a header is passed in as pattern, nor is an offset needed for a full search).

forwardbool, optional

Seek forward if True (default), backward if False.

maximumint, optional

Maximum number of bytes to search away from the present location. Default: search twice the frame size if given, otherwise 10000 (extra bytes to avoid partial patterns will be added). Use 0 to check only at the current position.

checkint or tuple of int, optional

Frame offsets where another header should be present. Default: 1, i.e., a sync pattern should be present one frame after the one found (independent of forward), thus helping to guarantee the frame is not corrupted.

Returns

headerVDIFHeader

Retrieved VDIF header.

Raises

~baseband.vlbi_base.base.HeaderNotFoundError

If no header could be located.

AssertionError

If the header did not pass verification.

get_frame_rate(self)

Determine the number of frames per second.

This method first tries to determine the frame rate by looking for the highest frame number in the first second of data. If that fails, it attempts to extract the sample rate from the header.

Returns

frame_rateQuantity

Frames per second.

get_thread_ids(self, check=2)

Determine the number of threads in the VDIF file.

The file is presumed to be positioned at the start of a header. Usually, it suffices to just seek to the start of the file, but if not, use find_header.

Parameters

checkint, optional

Number of extra frames to check. Frame sets are scanned until the number of thread IDs found no longer increases for check frames.

Returns

thread_idslist

Sorted list of all thread ids encountered in the frames scanned.

locate_frames(self, pattern, *, mask=None, frame_nbytes=None, offset=0, forward=True, maximum=None, check=1)

Use a pattern to locate frame starts near the current position.

Note that the current position is always included.

Parameters

patternheader, ~numpy.ndaray, bytes, int, or iterable of int

Synchronization pattern to look for. If a header or header class, invariant_pattern() is used to create a masked pattern, using invariant keys from invariants(). If an ndarray or bytes instance, a byte array view is taken. If an (iterable of) int, the integers need to be unsigned 32 bit and will be interpreted as little-endian.

mask~numpy.ndarray, bytes, int, or iterable of int.

Bit mask for the pattern, with 1 indicating a given bit will be used the comparison.

frame_nbytesint, optional

Frame size in bytes. Defaults to the frame size in any header passed in.

offsetint, optional

Offset from the frame start that the pattern occurs. Any offsets inferred from masked entries are added to this (hence, no offset needed when a header is passed in as pattern).

forwardbool, optional

Seek forward if True (default), backward if False.

maximumint, optional

Maximum number of bytes to search away from the present location. Default: search twice the frame size if given, otherwise 1 million (extra bytes to avoid partial patterns will be added). Use 0 to check only at the current position.

checkint or tuple of int, optional

Frame offsets where another sync pattern should be present (if inside the file). Ignored if frame_nbytes is not given. Default: 1, i.e., a sync pattern should be present one frame after the one found (independent of forward), thus helping to guarantee the frame is not corrupted.

Returns

locationslist of int

Locations of sync patterns within the range scanned, in order of proximity to the starting position.

read_frame(self, edv=None, verify=True)

Read a single frame (header plus payload).

Parameters

edvint, optional

The expected extended data version for the VDIF Header. If None, use that of the first frame. (Passing it in slightly improves file integrity checking.)

verifybool, optional

Whether to do basic checks of frame integrity. Default: True.

Returns

frameVDIFFrame

With .header and .data properties that return the VDIFHeader and data encoded in the frame, respectively.

read_frameset(self, thread_ids=None, edv=None, verify=True)

Read a single frame (header plus payload).

Parameters

thread_idslist, optional

The thread ids that should be read. If None (default), read all threads.

edvint, optional

The expected extended data version for the VDIF Header. If None, use that of the first frame. (Passing it in slightly improves file integrity checking.)

verifybool, optional

Whether to do basic checks of frame integrity. Default: True.

Returns

framesetVDIFFrameSet

With .headers and .data properties that return a list of VDIFHeader and the data encoded in the frame set, respectively.

read_header(self, edv=None, verify=True)

Read a single header from the file.

Parameters

edvint, False, or None, optional

Extended data version. If False, a legacy header is used. If None (default), it is determined from the header. (Given it explicitly is mostly useful for a slight speed-up.)

verifybool, optional

Whether to do basic verification of integrity. Default: True.

Returns

headerVDIFHeader

temporary_offset(self, offset=None, whence=0)

Context manager for temporarily seeking to another file position.

To be used as part of a with statement:

with fh_raw.temporary_offset() [as fh_raw]:
    with-block

On exiting the with-block, the file pointer is moved back to its original position. As a convenience, one can pass on the offset to seek to when entering the context manager. Parameters are as for io.IOBase.seek().