SwissKnife package

Submodules

SwissKnife.architectures module

SwissKnife.architectures.SkipConNet(x_train, dropout)

Args:

x_train: dropout:

SwissKnife.architectures.classification_large(input_shape)

Args:

input_shape:

SwissKnife.architectures.classification_scratch(input_shape)

Args:

input_shape:

SwissKnife.architectures.classification_small(input_shape, num_classes)

Args:

input_shape: num_classes:

SwissKnife.architectures.dlc_model(input_shape, num_classes)

Model for classification on top of pose estimation.

Classification model for behavior, operating on pose estimation. This model has more free parameters than Sturman et al.

Parameters

• input_shape (keras compatible input shape (W,H,Channels)) – keras compatible input shape (features,)

• num_classes (int) – Number of behaviors to classify.

Returns

behavior (from pose estimates) model

Return type

keras.model

SwissKnife.architectures.dlc_model_sturman(input_shape, num_classes)

Model that implements behavioral classification based on Deeplabcut generated features as in Sturman et al.

Reimplementation of the model used in the publication Sturman et al. that performs action recognition on top of pose estimation

Parameters

• input_shape (keras compatible input shape (W,H,Channels)) – keras compatible input shape (features,)

• num_classes (int) – Number of behaviors to classify.

Returns

Sturman et al. model

Return type

keras.model

SwissKnife.architectures.idtracker_ai(input_shape, classes)

Implementation of the idtracker.ai identification module as described in the supplementary of Romero-Ferrero et al.

Parameters

• input_shape (keras compatible input shape (W,H,Channels)) – keras compatible input shape (features,)

• num_classes (int) – Number of behaviors to classify..

Returns

idtracker.ai identification module

Return type

keras.model

SwissKnife.architectures.posenet(input_shape, num_classes, backbone='efficientnetb5', fix_backbone=False, gaussian_noise=0.05, features=256, bias=False)

Model that implements SIPEC:PoseNet architecture.

This model uses an EfficientNet backbone and deconvolves generated features into landmarks in imagespace. It operates on single images and can be used in conjuntion with SIPEC:SegNet to perform top-down pose estimation.

Parameters

• input_shape (keras compatible input shape (W,H,Channels)) – keras compatible input shape (features,)

• num_classes (int) – Number of joints/landmarks to detect.

• backbone (str) – Backbone/feature detector to use, default is EfficientNet5. Choose smaller/bigger backbone depending on GPU memory.

• gaussian_noise (float) – Kernel size of gaussian noise layers to use.

• features (int) – Number of feature maps to generate at each level.

• bias (bool) – Use bias for deconvolutional layers.

Returns

SIPEC:PoseNet

Return type

keras.model

SwissKnife.architectures.pretrained_recognition(model_name, input_shape, num_classes, fix_layers=True)

This returns the model architecture for a model that operates on images and is pretrained with imagenet weights. This architecture is used for IdNet and BehaviorNet as backbone in SIPEC and is referred to as RecognitionNet.

Parameters

• model_name (keras.model) – Name of the pretrained recognition model to use (names include: “xception, “resnet”, “densenet”)

• input_shape (np.ndarray - (Time, Width, Height, Channels)) – Shape of the images over time.

• num_classes (int) – Number of behaviors to recognise.

• fix_layers (bool) – Recurrent dropout factor to use.

Returns

RecognitionNet

Return type

keras.model

SwissKnife.architectures.recurrent_model_lstm(recognition_model, recurrent_input_shape, classes=4, recurrent_dropout=None)

Recurrent architecture for classification of temporal sequences of images based on LSTMs or GRUs. This architecture is used for IdNet in SIPEC.

Parameters

• recognition_model (keras.model) – Pretrained recognition model that extracts features for individual frames.

• recurrent_input_shape (np.ndarray - (Time, Width, Height, Channels)) – Shape of the images over time.

• classes (int) – Number of behaviors to recognise.

• recurrent_dropout (float) – Recurrent dropout factor to use.

Returns

IdNet

Return type

keras.model

SwissKnife.architectures.recurrent_model_old(recognition_model, recurrent_input_shape, classes=4, recurrent_dropout=None)

Args:

recognition_model: recurrent_input_shape: classes: recurrent_dropout:

SwissKnife.architectures.recurrent_model_tcn(recognition_model, recurrent_input_shape, classes=4)

Recurrent architecture for classification of temporal sequences of images based on temporal convolution architecture (TCN). This architecture is used for BehaviorNet in SIPEC.

Parameters

• recognition_model (keras.model) – Pretrained recognition model that extracts features for individual frames.

• recurrent_input_shape (np.ndarray - (Time, Width, Height, Channels)) – Shape of the images over time.

• classes (int) – Number of behaviors to recognise.

Returns

BehaviorNet

Return type

keras.model

SwissKnife.augmentations module

SwissKnife.augmentations.mouse_identification(level=2)

Augmentation for mouse identification.

This functions returns an imgaug augmentation object according, where the strength can be controlled by some integers values called level. This augmentaiton object is directly usable with one of SIPEC’s networks during training.

Parameters

level (int) – Level of augmentation, higher value indicates stronger image manipulations.

Returns

augmenter

Return type

imgaug.augmenters

SwissKnife.augmentations.mouse_poseestimation()

SwissKnife.augmentations.primate_identification(level=2)

Augmentation for primate identification.

This functions returns an imgaug augmentation object according, where the strength can be controlled by some integers values called level. This augmentaiton object is directly usable with one of SIPEC’s networks during training.

Parameters

level (int) – Level of augmentation, higher value indicates stronger image manipulations.

Returns

augmenter

Return type

imgaug.augmenters

SwissKnife.augmentations.primate_poseestimation()

SwissKnife.behavior module

SwissKnife.classification_comparison module

SwissKnife.classification_comparison.main()

SwissKnife.classification_comparison.remove_layers(model, num_layers)

SwissKnife.classification_comparison.run_experiment(base_path, config, num_classes=4, results_sink='', continuation=0, fraction=None)

SwissKnife.dataloader module

class SwissKnife.dataloader.Dataloader(x_train, y_train, x_test, y_test, config, with_dlc=False, dlc_train=None, dlc_test=None)

Bases: object

categorize_data(num_classes, recurrent=False)

Args:

num_classes: recurrent:

change_dtype()

create_dataset(oneD, look_back=5)

Args:

oneD: look_back:

create_flattened_data()

create_recurrent_data(oneD=False, recurrent_labels=True)

Args:

oneD:

create_recurrent_data_dlc(recurrent_labels=True)

create_recurrent_labels()

decimate_labels(percentage, balanced=False)

decimate labels to a given percentate percentage in [0,1] :return:

Args:

percentage: balanced:

decode_labels(labels)

Args:

labels:

downscale_frames(factor=0.5)

encode_label(label)

Args:

label:

encode_labels()

expand_dims()

get_input_shape(recurrent=False)

Args:

recurrent:

normalize_data()

prepare_data(downscale=0.5, remove_behaviors=[], flatten=False)

reduce_labels(behavior, num_labels)

Args:

behavior: num_labels:

remove_behavior(behavior)

Args:

behavior:

undersample_data()

SwissKnife.dataloader.create_dataset(dataset, look_back=5, oneD=False)

Create a recurrent dataset from array.

Parameters

• dataset (np.ndarray) – numpy array of dataset to make recurrent

• look_back (int) – Number of timesteps to look into the past and future.

• oneD (bool) – Boolean that indicates if the current dataset is one dimensional.

Returns

recurrent dataset

Return type

np.ndarray

SwissKnife.dataprep module

class SwissKnife.dataprep.Dataset(species)

Bases: SwissKnife.mrcnn.utils.Dataset

image_reference(image_id)

Return the path of the image.

load(dataset_dir, subset, annotations)

Load a subset of the Balloon dataset. dataset_dir: Root directory of the dataset. subset: Subset to load: train or val

load_mask(image_id)

Generate instance masks for an image. Returns:

masks: A bool array of shape [height, width, instance count] with

one mask per instance.

class_ids: a 1D array of class IDs of the instance masks.

SwissKnife.dataprep.generate_individual_mouse_data(animal_lim=None, cv_folds=5, fold=0, day=1, masking=False)

SwissKnife.dataprep.generate_mouse_pose_data()

SwissKnife.dataprep.generate_primate_pose_data()

SwissKnife.dataprep.get_SIPEC_reproduction_data(name)

SwissKnife.dataprep.get_individual_mouse_data()

SwissKnife.dataprep.get_mouse_dlc_data()

SwissKnife.dataprep.get_mouse_pose_data(fraction=1.0)

SwissKnife.dataprep.get_mouse_pose_dlc_comparison_data(fold)

SwissKnife.dataprep.get_primate_identification_data(scaled=True)

SwissKnife.dataprep.get_primate_pose_data()

SwissKnife.dataprep.get_segmentation_data(frames_path=None, annotations_path=None, name=None, fold=None, cv_folds=None, fraction=None)

SwissKnife.dataprep.main()

SwissKnife.dataprep.prepareData(frames_path, annotations_path, species, fold=None, cv_folds=None, fraction=None)

SwissKnife.extract_videos module

SwissKnife.full_inference module

SwissKnife.full_inference.full_inference(videodata, results_sink, networks, example_frame, id_classes, mask_matching=False, id_matching=False, mask_size=256, lookback=100, mold_dimension=1024, max_ids=4)

Performs full inference on a given video using available SIPEC modules.

Parameters

• videodata (np.ndarray) – numpy array of read-in videodata.

• results_sink (str) – Path to where data will be saved.

• networks (dict) – Dictionary containing SIPEC modules to be used for full inference (“SegNet”, “PoseNet”, “BehaveNet”, IdNet”)

• mask_matching (bool) – Use greedy-mask-matching

• id_matching (bool) – Correct/smooth SIPEC:IdNet identity using identities based on temporal tracking (greedy-mask-matching)

• mask_size (int) – Mask size used for the cutout of animals.

• lookback (int) – Number of timesteps to look back into the past for id_matching.

• max_ids (int) – Number of maximum ids / maximum number of animals in any FOV.

Returns

Outputs of all the provided SIPEC modules for each video frame.

Return type

list

SwissKnife.full_inference.main()

SwissKnife.identification module

SwissKnife.job_runner module

SwissKnife.job_runner.main()

SwissKnife.masksmoothing module

class SwissKnife.masksmoothing.MaskMatcher(max_ids=4)

Bases: object

MaskMatcher class to match segmentation masks of temporally adjacent frames.

This class implements extensions of greedy masks matching algorithms.

assign_ids(frame_cur, _model, _classes, _threshold, bboxes_cur, bboxes_pre, ids_pre)

Identify all bboxes in the current frame. Call IdNet for unmatched bboxes. In case of repeated ids, label the less probable ones as “Wrong”. Output: a list of tuples, each tuple is (ID, confidence level).

bbox_match(bboxes_cur, bboxes_pre)

Bounding box greedy matching algorithm.

Find all current bboxes which are identical to any in the previous frame. Condition: a bbox_cur intersects with one and only bbox_pre && it doesn’t intersect with any other bbox_cur.

Parameters

• bboxes_cur (bounding boxes) – Bounding boxes from current frame.

• bboxes_pre (bounding boxes) – Bounding boxes from previous frame.

Returns

Dictionary containing the mapping of indices from matching bounding boxes of current frame to previous frame.

Return type

dict

bbox_to_polygon(bbox)

iou(bbox1, bbox2)

Calculate the intersection over union (IoU) for two bounding boxes.

Parameters

• bbox1 (bounding box) – First bounding box for IoU calculation.

• bbox2 (bounding box) – Second bounding box for IoU calculation.

Returns

IoU value.

Return type

float

map(mapping, arr)

match_ids(mapping, nums)

match_masks(bboxes_cur, bboxes_pre)

SwissKnife.masksmoothing.main()

SwissKnife.model module

class SwissKnife.model.Model(config=None)

Bases: object

add_callbacks(callbacks)

Args:

callbacks:

export_training_details()

fix_recognition_layers(num=None)

Args:

num:

load_recognition_model(path)

Loads recognition model. Args:

path: Path to existing recognition model.

predict(data, model='recognition', threshold=None)

Args:

data: model: threshold:

predict_sequential(data)

Args:

data:

remove_classification_layers()

save_model()

scheduler(epoch)

Args:

epoch:

set_augmentation(augmentation)

Args:

augmentation:

set_class_weight(class_weight)

Args:

class_weight:

set_lr_scheduler()

set_optimizer(name, lr=0.00035)

Args:

name: lr:

set_recognition_model(architecture, input_shape, num_classes)

Sets recognitions model. Args:

architecture: input_shape: num_classes:

set_sequential_model(architecture, input_shape, num_classes)

Args:

architecture: input_shape: num_classes:

train_recognition_network(dataloader)

Args:

dataloader:

train_sequential_network(dataloader)

Args:

dataloader:

update_params(config)

Args:

config:

SwissKnife.poseestimation module

class SwissKnife.poseestimation.DataGenerator(imgs, masks, augmentation, batch_size=32, dim=(32, 32, 32), n_channels=1, shuffle=True)

Bases: tensorflow.python.keras.utils.data_utils.Sequence

Generates data for Keras

on_epoch_end()

Updates indexes after each epoch

class SwissKnife.poseestimation.Metrics(writer=None, unmold=None)

Bases: tensorflow.python.keras.callbacks.Callback

get_data()

on_epoch_end(batch, logs={})

Called at the end of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Arguments:

epoch: Integer, index of epoch. logs: Dict, metric results for this training epoch, and for the

validation epoch if validation is performed. Validation result keys are prefixed with val_. For training epoch, the values of the

Model’s metrics are returned. Example : {‘loss’: 0.2, ‘acc’: 0.7}.

on_train_begin(logs={})

Called at the beginning of training.

Subclasses should override for any actions to run.

Arguments:

logs: Dict. Currently no data is passed to this argument for this method

but that may change in the future.

setModel(model)

SwissKnife.poseestimation.calculate_rmse(pred, true)

Calculate Root Mean Squared Error (RMSE)

Calculate RMSE between predicted and ground truth landmarks for pose estimation.

Parameters

• pred (np.ndarray) – Coordinates of predicted landmarks of pose estimation network.

• true (np.ndarray) – Coordinates of ground truth landmarks of pose estimation network.

Returns

model

Return type

keras.model

class SwissKnife.poseestimation.callbacks_viz_poseestimation

Bases: tensorflow.python.keras.callbacks.Callback

on_epoch_end(batch, logs={})

Called at the end of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Arguments:

epoch: Integer, index of epoch. logs: Dict, metric results for this training epoch, and for the

validation epoch if validation is performed. Validation result keys are prefixed with val_. For training epoch, the values of the

Model’s metrics are returned. Example : {‘loss’: 0.2, ‘acc’: 0.7}.

on_train_begin(logs={})

Called at the beginning of training.

Subclasses should override for any actions to run.

Arguments:

logs: Dict. Currently no data is passed to this argument for this method

but that may change in the future.

setModel(model)

SwissKnife.poseestimation.custom_binary_crossentropy(y_true, y_pred, from_logits=False, label_smoothing=0)

SwissKnife.poseestimation.evaluate_pose_estimation(x_test, y_test, posenet, remold=False, y_test_orig=None, x_test_orig=None, coms_test=None)

SwissKnife.poseestimation.main()

SwissKnife.poseestimation.read_DLC_data(dlc_path, folder, label_file_path, exclude_labels=[], as_gray=False)

SwissKnife.poseestimation.read_dlc_labels_from_folder(dlc_path, exclude_labels=[])

SwissKnife.poseestimation.revert_mold(img, padding, scale, dtype='uint8')

class SwissKnife.poseestimation.rmse_metric

Bases: tensorflow.python.keras.callbacks.Callback

get_data()

on_epoch_end(batch, logs={})

Called at the end of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Arguments:

epoch: Integer, index of epoch. logs: Dict, metric results for this training epoch, and for the

validation epoch if validation is performed. Validation result keys are prefixed with val_. For training epoch, the values of the

Model’s metrics are returned. Example : {‘loss’: 0.2, ‘acc’: 0.7}.

on_train_begin(logs={})

Called at the beginning of training.

Subclasses should override for any actions to run.

Arguments:

logs: Dict. Currently no data is passed to this argument for this method

but that may change in the future.

setModel(model)

SwissKnife.poseestimation.segment_images_and_masks(X, y, SegNet, asgrey=False, mask_size=64)

SwissKnife.poseestimation.train_on_data(x_train, y_train, x_test, y_test, config, results_sink, segnet_path=None, augmentation='primate', original_img_shape=None, save=None)

SwissKnife.poseestimation.treshold_maps(y, threshold=0.9)

SwissKnife.poseestimation.vis_locs(X, y)

SwissKnife.poseestimation.vis_maps(X, y)

SwissKnife.segmentation module

segmentation.py

The core module of my example project

class SwissKnife.segmentation.IneichenConfig

Bases: SwissKnife.mrcnn.config.Config

BATCH_SIZE = 1

DETECTION_MIN_CONFIDENCE = 0.95

GPU_COUNT = 1

IMAGES_PER_GPU = 1

IMAGE_MAX_DIM = 320

IMAGE_MIN_DIM = 320

IMAGE_RESIZE_MODE = 'square'

MINI_MASK_SHAPE = (56, 56)

NAME = 'mouse'

NUM_CLASSES = 2

STEPS_PER_EPOCH = 100

class SwissKnife.segmentation.InferencIneichenConfig

Bases: SwissKnife.segmentation.IneichenConfig

BATCH_SIZE = 1

DETECTION_MIN_CONFIDENCE = 0.99

IMAGES_PER_GPU = 1

class SwissKnife.segmentation.InferenceConfigMouse

Bases: SwissKnife.segmentation.MouseConfig

BATCH_SIZE = 1

DETECTION_MIN_CONFIDENCE = 0.9

IMAGES_PER_GPU = 1

class SwissKnife.segmentation.InferenceConfigPrimate

Bases: SwissKnife.segmentation.PrimateConfig

BATCH_SIZE = 1

DETECTION_MIN_CONFIDENCE = 0.8

IMAGES_PER_GPU = 1

IMAGE_MAX_DIM = 1920

IMAGE_MIN_DIM = 1920

IMAGE_RESIZE_MODE = 'square'

IMAGE_SHAPE = [1920, 1920, 3]

class SwissKnife.segmentation.MaskFilter

Bases: object

Return the most important thing about a person. :Parameters: your_name – A string indicating the name of the person.

predict()

train()

class SwissKnife.segmentation.MouseConfig

Bases: SwissKnife.mrcnn.config.Config

BACKBONE = 'resnet101'

BATCH_SIZE = 1

DETECTION_MIN_CONFIDENCE = 0.5

GPU_COUNT = 1

GRADIENT_CLIP_NORM = 1.0

IMAGES_PER_GPU = 1

IMAGE_MAX_DIM = 1024

IMAGE_MIN_DIM = 1024

IMAGE_RESIZE_MODE = 'square'

IMAGE_SHAPE = [1024, 1024, 3]

LEARNING_RATE = 0.001

MAX_GT_INSTANCES = 4

MINI_MASK_SHAPE = (56, 56)

NAME = 'mouse'

NUM_CLASSES = 2

STEPS_PER_EPOCH = 100

TRAIN_ROIS_PER_IMAGE = 128

USE_MINI_MASK = True

WEIGHT_DECAY = 0.0001

class SwissKnife.segmentation.PrimateConfig

Bases: SwissKnife.mrcnn.config.Config

BACKBONE = 'resnet101'

BATCH_SIZE = 2

DETECTION_MAX_INSTANCES = 10

DETECTION_MIN_CONFIDENCE = 0.85

GPU_COUNT = 1

GRADIENT_CLIP_NORM = 1.0

IMAGES_PER_GPU = 2

IMAGE_MAX_DIM = 1280

IMAGE_MIN_DIM = 1280

IMAGE_RESIZE_MODE = 'crop'

IMAGE_SHAPE = [1280, 1280, 3]

LEARNING_RATE = 0.0025

MAX_GT_INSTANCES = 4

MINI_MASK_SHAPE = (56, 56)

NAME = 'primate'

NUM_CLASSES = 2

STEPS_PER_EPOCH = 100

TRAIN_ROIS_PER_IMAGE = 200

USE_MINI_MASK = True

WEIGHT_DECAY = 0.0001

class SwissKnife.segmentation.SegModel(species)

Bases: object

detect_image(img, mold=True, verbose=1)

Args:

img: mold: verbose:

detect_image_original(img, mold=True, verbose=1)

Args:

img: mold: verbose:

detect_video(video, results_sink=None)

Args:

video: results_sink:

evaluate(dataset_val, maskfilter=None)

Evaluate segmentation model on a given validation set. Args:

dataset_val:Validation dataset. maskfilter:

init_augmentation()

Initializes the augmentation for segmentation network training, different default levels are available. Args:

dataset_train: dataset_val:

init_training(model_path, init_with='coco')

Initialized training of a new or existing segmentation network. Args:

model_path:Path to segmentation network, either existing or desired path for new model. init_with:The initializations “imagenet” or “coco” are available for new segmentation models and “last” if retraining existing network.

set_inference(model_path=None)

Set segmentation model to inference. Args:

model_path:

train(dataset_train, dataset_val)

Train the segmentation network. Args:

dataset_train: dataset_val:

SwissKnife.segmentation.do_ablation(species, cv_folds, random_seed, fraction)

Args:

species: cv_folds: random_seed: fraction:

SwissKnife.segmentation.evaluate_network(model_path, species, filter_masks=False, cv_folds=0)

Args:

model_path: species: filter_masks: cv_folds:

SwissKnife.segmentation.main()

SwissKnife.segmentation.mold_image(img, config=None, dimension=None, min_dimension=None, return_all=False)

Args:

img: config: dimension:

SwissKnife.segmentation.mold_video(video, dimension, n_jobs=40)

Args:

video: dimension: n_jobs:

SwissKnife.segmentation.train_on_data(species, cv_folds, fraction=None, to_file=True, experiment='', fold=None)

Args:

species: cv_folds: fraction: to_file: experiment: fold:

SwissKnife.segmentation.train_on_data_once(model_path, cv_folds=0, frames_path=None, annotations_path=None, species=None, fold=0, fraction=None, perform_evaluation=True, debug=0)

Performs training for the segmentation moduel of SIPEC (SIPEC:SegNet).

Parameters

• model_path (str) – Path to model, can be either where a new model should be stored or a path to an existing model to be retrained.

• cv_folds (int) – Number of cross_validation folds, use 0 for a normal train/test split.

• frames_path (str) – Path to the frames used for training.

• annotations_path (str) – Path to the annotations used for training.

• species (str) – Species to perform segmentation on (can be any species, but “mouse” or “primate” have more specialised parameters). If your species is neither “mouse” nor “primate”, use “default”.

• fold (int) – If cv_folds > 1, fold is the number of fold to be tested on.

• fraction (float) – Factor by which to decimate the training data points.

• perform_evaluation (bool) – Perform subsequent evaluation of the model

• debug (bool) – Debug verbosity.

Returns

• model – SIPEC:SegNet model

• mean_ap – Mean average precision score achieved by this model

SwissKnife.segmentation.train_on_data_path(annotations, frames)

Args:

annotations: frames:

SwissKnife.test module

SwissKnife.unsupervised module

SwissKnife.utils module

class SwissKnife.utils.Metrics(validation_data)

Bases: tensorflow.python.keras.callbacks.Callback

get_data()

on_epoch_end(batch, logs={})

Called at the end of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Arguments:

epoch: Integer, index of epoch. logs: Dict, metric results for this training epoch, and for the

validation epoch if validation is performed. Validation result keys are prefixed with val_. For training epoch, the values of the

Model’s metrics are returned. Example : {‘loss’: 0.2, ‘acc’: 0.7}.

on_train_begin(logs={})

Called at the beginning of training.

Subclasses should override for any actions to run.

Arguments:

logs: Dict. Currently no data is passed to this argument for this method

but that may change in the future.

setModel(model)

class SwissKnife.utils.ResultsTracker(path=None)

Bases: object

add_result(results)

file_available()

write_results(results)

SwissKnife.utils.apply_all_masks(masks, coms, img, mask_size=128)

SwissKnife.utils.apply_to_mask(mask, img, com, mask_size)

SwissKnife.utils.balanced_acc(y_true, y_pred)

SwissKnife.utils.bbox_mask(model, img, verbose=0)

SwissKnife.utils.calculate_speed(distances)

SwissKnife.utils.callbacks_learningRate_plateau()

SwissKnife.utils.callbacks_tf_logging(path='./logs/')

SwissKnife.utils.categorical_focal_loss(gamma=2.0, alpha=0.25)

Implementation of Focal Loss from the paper in multiclass classification Formula:

loss = -alpha*((1-p)^gamma)*log(p)

Parameters:

alpha – the same as wighting factor in balanced cross entropy gamma – focusing parameter for modulating factor (1-p)

Default value:

gamma – 2.0 as mentioned in the paper alpha – 0.25 as mentioned in the paper

SwissKnife.utils.check_directory(directory)

SwissKnife.utils.check_folder(directory)

SwissKnife.utils.clearMemory(model, backend)

SwissKnife.utils.coords_to_masks(coords, dim=2048)

SwissKnife.utils.crop_pngs()

SwissKnife.utils.detect_primate(_img, _model, classes, threshold)

SwissKnife.utils.dilate_mask(mask, factor=20)

SwissKnife.utils.distance(x, y, x_prev, y_prev)

SwissKnife.utils.eval_model(model, data, results_dict, results_array, filename, dataloader, model_name='')

SwissKnife.utils.extractCOM(image, threshold)

SwissKnife.utils.extractCOM_only(image)

SwissKnife.utils.f1(y_true, y_pred)

SwissKnife.utils.f1_loss(y_true, y_pred)

SwissKnife.utils.fix_layers(network, with_backbone=True)

SwissKnife.utils.get_ax(rows=1, cols=1, size=8)

Return a Matplotlib Axes array to be used in all visualizations in the notebook. Provide a central point to control graph sizes.

Change the default size attribute to control the size of rendered images

SwissKnife.utils.get_optimizer(optim_name, lr=0.01)

SwissKnife.utils.get_tensorbaord_callback(path='./logs')

SwissKnife.utils.heatmap_mask(maps, mask)

SwissKnife.utils.heatmap_to_scatter(heatmaps, threshold=6e-10)

SwissKnife.utils.heatmaps_for_image(labels, window=100, sigma=3)

SwissKnife.utils.heatmaps_for_image_whole(labels, img_shape, sigma=3, threshold=None)

SwissKnife.utils.heatmaps_for_images(labels, img_shape, sigma=3, threshold=None)

SwissKnife.utils.heatmaps_to_locs(y)

SwissKnife.utils.keypoints_in_mask(mask, keypoints)

SwissKnife.utils.loadVideo(path, num_frames=None, greyscale=True)

SwissKnife.utils.load_config(path)

SwissKnife.utils.load_dict(filename)

SwissKnife.utils.load_vgg_labels(annotations, video_length, framerate_video, behavior=None)

SwissKnife.utils.mask_to_original_image(orig_shape, mask, center_of_mass, mask_size)

SwissKnife.utils.maskedImg(img, center_of_mass, mask_size=74)

SwissKnife.utils.masks_to_coms(masks)

SwissKnife.utils.masks_to_coords(masks)

SwissKnife.utils.pathForFile(paths, filename)

SwissKnife.utils.plotHistory(history, measure)

SwissKnife.utils.rescale_img(mask, frame, mask_size=256)

SwissKnife.utils.resize(image, output_shape, order=1, mode='constant', cval=0, clip=True, preserve_range=False, anti_aliasing=False, anti_aliasing_sigma=None)

A wrapper for Scikit-Image resize().

Scikit-Image generates warnings on every call to resize() if it doesn’t receive the right parameters. The right parameters depend on the version of skimage. This solves the problem by using different parameters per version. And it provides a central place to control resizing defaults.

SwissKnife.utils.resize_image(image, min_dim=None, max_dim=None, min_scale=None, mode='square')

SwissKnife.utils.run_ai_cumulative_gradient(optimizer)

SwissKnife.utils.saveModel(model)

SwissKnife.utils.save_dict(filename, dict)

SwissKnife.utils.setGPU(gpu_name, growth=True)

SwissKnife.utils.setGPU_growth()

SwissKnife.utils.set_random_seed(random_seed)

SwissKnife.utils.startend(df_entry, ms, df)

SwissKnife.utils.train_model(model, optimizer, epochs, batch_size, data_train, data_val=None, callbacks=None, class_weights=None, loss='crossentropy', augmentation=None, num_gpus=1)

SwissKnife.visualization module

SwissKnife.visualization.apply_mask(image, mask, color, alpha=0.5)

Apply the given mask to the image.

SwissKnife.visualization.displayBoxes(frame, mask, color=(0, 0, 255), animal_id=None, mask_id=None)

SwissKnife.visualization.displayScatter(frame, coords, color=(0, 0, 255))

SwissKnife.visualization.main()

SwissKnife.visualization.visualize_full_inference(results_sink, networks, video, results, output_video_name, display_coms=False, dimension=1024)

SwissKnife.visualization.visualize_labels_on_video(video_path, labels_path, outpath)

SwissKnife.visualization.visualize_labels_on_video_cv(video, labels, framerate_video, out_path)

Module contents