_utils

cotk._utils often is used by internal lib. The users should not find api here.

cotk._utils.trim_before_target(lists, target)

Trim the list before the target. If there is no target, return the origin list.

Parameters

• lists (list)

• target

cotk._utils.chain_sessions(sessions) → Tuple[List[Any], List[int]]

cotk._utils.restore_sessions(chained_sessions, session_lengths) → List[List[Any]]

cotk._utils.replace_unk(sentences, unk_token, target_token=-1)

Auxiliary function for replacing the unknown words to another words

Parameters

• input (list[List[Any]]) – the sentences

• unk_tokens (Any) – id for unknown words.

• target – the target word index used to replace the unknown words.

Returns

** list* – processed result.

imports

cotk._utils.imports provides classes that fake a uninstalled module.

class cotk._utils.imports.DummyObject(err)

Dummy Object for uninstalled modules. [deprecated]

Examples

>>> try:
...   import torch
... except ImportError as err:
...   torch = DummyObject(err)
...
>>> torch.Tensor
ModuleNotFoundError: No module named 'torch'
>>> torch.Tensor = DummyObject(torch.err)
>>> torch.Tensor
>>> torch.Tensor()
ModuleNotFoundError: No module named 'torch'

class cotk._utils.imports.LazyModule(module_name, global_dict)

Lazy loading modules.

Parameters

• module_name (str) – The path of import. For example: cotk.

• global_dict (str) – Override the global dictionary when the module is loaded.

Examples

>>> torch = LazyModule("torch", globals())
>>> print(torch)
<cotk._utils.imports.LazyModule object at 0x000001BE147682E8>
>>> torch.Tensor = LazyObject("torch.Tensor")
>>> print(torch.Tensor)
<cotk._utils.imports.LazyObject object at 0x000001BE1339CE80>
>>> print(torch.LongTensor)
<class 'torch.LongTensor'>
>>> print(torch.Tensor)
<class 'torch.Tensor'>
>>> print(torch)
<module 'torch'>

class cotk._utils.imports.LazyObject(object_name)

Lazy loading objects.

Parameters

object_name (str) – The path of import. For example: cotk.dataloader.MSCOCO.

Examples

>>> dataloader = LazyObject("cotk.dataloader")
>>> print(dataloader)
<cotk._utils.imports.LazyObject object at 0x000001BE1339CE48>
>>> print(dataloader.MSCOCO)
<class 'cotk.dataloader.language_generation.MSCOCO'>

metaclass

A lib for decorator and metaclass

class cotk._utils.metaclass.DocStringInheritor

A meta class. It make the class:

• Docstring can inherit the parent classes.

• {STRING_DOCS} in docs will be replaced by self.STRING_DOCS or BaseClass.STRING_DOCS. (choose the first available one)

• {BaseClassName.STRING_DOCS} in docs will be replaced by BaseClassName.STRING_DOCS. (BaseClassName can be itself)

• If the docstring iherited from the parent class Base, {_STRING_DOCS} try to use the value of self._STRING_DOCS first (if not existed, then trying BASE._STRING_DOCS), but {STRING_DOCS} (without underscore prefix) will use the value of Base.STRING_DOCS

• The replacement can be nested. But only {BaseClassName.STRING_DOCS} or {_STRING_DOCS} are allowed in the nested replacement, for avoiding ambiguous use.

A variation on http://groups.google.com/group/comp.lang.python/msg/26f7b4fcb4d66c95 by Paul McGuire from https://stackoverflow.com/questions/8100166/inheriting-methods-docstrings-in-python

class cotk._utils.metaclass.LoadClassInterface

The support of dynamic class load.

classmethod get_all_subclasses() → Iterable[Any]

Return a generator of all subclasses.

classmethod load_class(class_name) → Any

Return a subclass of class_name, case insensitively.

Parameters

class_name (str) – target class name.

unordered_hash

A module for hash unordered elements

class cotk._utils.unordered_hash.UnorderedSha256

Using SHA256 on unordered elements

update_data(data)

update digest by data. type(data)=bytes

update_hash(hashvalue)

update digest by hash. type(hashvalue)=bytes

digest() → bytes

return unordered hashvalue

hexdigest() → str

return unordered hashvalue

cotk._utils.unordered_hash.dumps(obj) → bytes

Generate bytes to identify the object by repr

cotk._utils.unordered_hash.dumps_json(obj) → bytes

Generate bytes to identify the object by json serialization