commonlibs.fileio package

Submodules

commonlibs.fileio.json module

JSON

class commonlibs.fileio.json.NDArrayEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: json.encoder.JSONEncoder

Serialise Numpy arrays

Use like this: json.dump(obj, fp, cls=NDArrayEncoder)

default(obj)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

commonlibs.fileio.json.dump_pretty_json(obj, fp, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=<class 'commonlibs.fileio.json.NDArrayEncoder'>, indent=4, separators=(', ', ': '), default=None, sort_keys=False, **kw)

Serialize obj as a JSON formatted stream to fp (a .write()-supporting file-like object).

If skipkeys is true then dict keys that are not basic types (str, int, float, bool, None) will be skipped instead of raising a TypeError.

If ensure_ascii is false, then the strings written to fp can contain non-ASCII characters if they appear in strings contained in obj. Otherwise, all such characters are escaped in JSON strings.

If check_circular is false, then the circular reference check for container types will be skipped and a circular reference will result in an OverflowError (or worse).

If allow_nan is false, then it will be a ValueError to serialize out of range float values (nan, inf, -inf) in strict compliance of the JSON specification, instead of using the JavaScript equivalents (NaN, Infinity, -Infinity).

If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation.

If specified, separators should be an (item_separator, key_separator) tuple. The default is (', ', ': ') if indent is None and (',', ': ') otherwise. To get the most compact JSON representation, you should specify (',', ':') to eliminate whitespace.

default(obj) is a function that should return a serializable version of obj or raise TypeError. The default simply raises TypeError.

If sort_keys is true (default: False), then the output of dictionaries will be sorted by key.

To use a custom JSONEncoder subclass (e.g. one that overrides the .default() method to serialize additional types), specify it with the cls kwarg; otherwise JSONEncoder is used.

commonlibs.fileio.paths module

Better file path handling

class commonlibs.fileio.paths.ProjectPaths(root_dir)

Bases: object

FORMATTER_COUNTER = 'counter'

UID_ROOT = 'root'

add_path(uid, path, uid_groups=None, is_absolute=False)

Add a path

Args:

uid:Unique identifier for the path path:Path string or Path() object uid_groups:Optional UID(s) to identify files by groups is_absolute:Flag indicating if given ‘path’ is absolute

add_subpath(uid_parent, uid, path, uid_groups=None)

Add a child path to an existing parent path

Args:

uid_parent:UID of the parent directory uid:UID of the new path path:relative path to add uid_groups:Optional UID(s) to identify files by groups

add_suffix(uid, new_suffix)

Add a suffix to a path

Args:

uid:Unique identifier for the path new_suffix:String with the new suffix

change_suffix(uid, new_suffix)

Modify the suffix of a path

Args:

uid:Unique identifier for the path new_suffix:String with the new suffix

counter

iter_group_paths(uid_groups, make_dirs=False, is_dir=False)

Return a generator with paths belong to group with given UID

Args:

uid_groups:Optional UID(s) to identify files by groups

make_dirs_for_groups(uid_groups, is_dir=True)

Create directories for all paths belonging to specified group(s)

Args:

uid_groups:Optional UID(s) to identify files by groups

remove_path(uid)

Remove a path from the bookkeeping

Args:

uid:Unique identifier for the path

rm_dirs_for_groups(uid_groups)

TODO

Args:

uid_groups:Optional UID(s) to identify files by groups

root

Return the Path() object for the project root directory

commonlibs.fileio.paths.join_paths(*paths)

Join two or more paths and return a new ‘Path()’ object

Args:

paths:Paths to join

Returns:

joined_path:Joined path

Raises:

TypeError:If not enough paths are given

Module contents

commonlibs.fileio.dump_pretty_json(obj, fp, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=<class 'commonlibs.fileio.json.NDArrayEncoder'>, indent=4, separators=(', ', ': '), default=None, sort_keys=False, **kw)

Serialize obj as a JSON formatted stream to fp (a .write()-supporting file-like object).

If skipkeys is true then dict keys that are not basic types (str, int, float, bool, None) will be skipped instead of raising a TypeError.

If ensure_ascii is false, then the strings written to fp can contain non-ASCII characters if they appear in strings contained in obj. Otherwise, all such characters are escaped in JSON strings.

If check_circular is false, then the circular reference check for container types will be skipped and a circular reference will result in an OverflowError (or worse).

If allow_nan is false, then it will be a ValueError to serialize out of range float values (nan, inf, -inf) in strict compliance of the JSON specification, instead of using the JavaScript equivalents (NaN, Infinity, -Infinity).

If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation.

If specified, separators should be an (item_separator, key_separator) tuple. The default is (', ', ': ') if indent is None and (',', ': ') otherwise. To get the most compact JSON representation, you should specify (',', ':') to eliminate whitespace.

default(obj) is a function that should return a serializable version of obj or raise TypeError. The default simply raises TypeError.

If sort_keys is true (default: False), then the output of dictionaries will be sorted by key.

To use a custom JSONEncoder subclass (e.g. one that overrides the .default() method to serialize additional types), specify it with the cls kwarg; otherwise JSONEncoder is used.