File structure management utils

This module contains all classes, that work with file structure

• FileStructManager provide all modules registration
• CheckpointsManager provide checkpoints management

class piepline.utils.fsm.FileStructManager(base_dir: str, is_continue: bool, exists_ok: bool = False)

Class, that provide directories registration in base directory.

All modules, that use file structure under base directory should register their paths in this class by pass module to method register_dir(). If directory also registered registration method will raise exception FSMException

Parameters:

• base_dir – path to directory with checkpoints
• is_continue – is FileStructManager used for continue training or predict
• exists_ok – if True - all checks for existing directories will be disabled

exception FSMException(message: str)

get_path(obj: piepline.utils.fsm.FolderRegistrable, create_if_non_exists: bool = False, check: bool = True) → str

Get path of registered object

Parameters:

• obj – object
• create_if_non_exists – is need to create object’s directory if it doesn’t exists
• check – is need to check object’s directory existing

Returns:

path to directory

Raises:

FSMException – if directory exists and check == True

in_continue_mode() → bool

Is FileStructManager in continue mode

Returns:True if in continue

register_dir(obj: piepline.utils.fsm.FolderRegistrable, check_name_registered: bool = False, check_dir_registered: bool = True) → None

Register directory in file structure

Parameters:

• obj – object to registration
• check_name_registered – is need to check if object name also registered
• check_dir_registered – is need to check if object path also registered

Raises:

FileStructManager – if path or object name also registered and if path also exists (in depends of optional parameters values)

class piepline.utils.fsm.FolderRegistrable(fsm: piepline.utils.fsm.FileStructManager)

Abstract class for implement classes, that use folders

Parameters:fsm – FileStructureManager class instance

class piepline.utils.fsm.MultipleFSM(base_dir: str, is_continue: bool, exists_ok: bool = False)