Media processing engines

inaFaceAnalyzer module implements four analysis engines allowing to process video or image streams :

Media analyzer classes share a common interface inherited from abstract class FaceAnalyzer. They are designed as *functions objects* or *functors* and can be used as functions, executing the code implemented in __call__ methods, with first argument corresponding to the media to analyze and returning pandas.DataFrame

Custom face detection, face classifier, eye detection and image preprocessing strategies can be provided in the constructor.

>>> from inaFaceAnalyzer.inaFaceAnalyzer import VideoAnalyzer
>>> # a video analyzer instance with default parameters
>>> va = VideoAnalyzer()
>>> df = va(sample_vid)
class inaFaceAnalyzer.inaFaceAnalyzer.FaceAnalyzer(face_detector=None, face_classifier=None, batch_len=32, verbose=False)[source]

Bases: ABC

This is an abstract class containg the pipeline used to process images, videos, with/without tracking * image/video decoding * face detection * face tracking (optional) * eye detection * face preprocessing * face classification

abstract __call__(src)[source]

Method to be implemented by each analyzer

Parameters:

src (str or list) – path to the video/image to be analyzed May also be a list of images

Return type:

Results stored in a pandas.DataFrame

__init__(face_detector=None, face_classifier=None, batch_len=32, verbose=False)[source]
Construct a face processing pipeline composed of a face detector,

a face preprocessing strategy and a face classifier. The face preprocessing strategy is defined based on the classifier’s properties.

Parameters:
__weakref__

list of weak references to the object (if defined)

class inaFaceAnalyzer.inaFaceAnalyzer.ImageAnalyzer(face_detector=None, face_classifier=None, batch_len=32, verbose=False)[source]

Bases: FaceAnalyzer

ImageAnalyzer instances allow to detect and classify faces from images

frame

bbox

detect_conf

sex_decfunc

age_decfunc

sex_label

age_label

0

./media/dknuth.jpg

(81, 52, 320, 291)

0.999999

7.24841

6.68495

m

61.8495

1

./media/800px-India_(236650352).jpg

(193, 194, 494, 495)

1

9.96501

5.76855

m

52.6855

2

./media/800px-India_(236650352).jpg

(472, 113, 694, 336)

0.999992

15.1933

4.09797

m

35.9797

3

./media/800px-India_(236650352).jpg

(40, 32, 109, 101)

0.999967

11.3448

4.35364

m

38.5364

4

./media/800px-India_(236650352).jpg

(384, 54, 458, 127)

0.999964

11.3798

4.36526

m

38.6526

5

./media/800px-India_(236650352).jpg

(217, 67, 301, 151)

0.999899

9.78476

4.8296

m

43.296
__call__(img_paths)[source]
Parameters:

img_paths (str or list) – path or list of paths to image file(s) to analyze

Returns:

  • pandas Dataframe with column ‘frame’ containing the path to the source

  • image. Remaining columns depend on processing options selected and

  • contain bounding box, and face classification information

class inaFaceAnalyzer.inaFaceAnalyzer.VideoAnalyzer(face_detector=None, face_classifier=None, batch_len=32, verbose=False)[source]

Bases: FaceAnalyzer

Video Analyzer allows to detect and classify faces in video streams

__call__(video_path, fps=None, offset=0)[source]

Pipeline function for face classification from videos (without tracking)

Parameters:

  • video_path – str Path to input video.

  • fps – float or None (default) amount of video frames to process per seconds if set to None, all frames are processed (costly)

  • offset – float (default: 0) Time in milliseconds to skip at the beginning of the video.

Returns:

frame position, coordinates, predictions, decision function,labels…

Return type:

Dataframe with frame and face information

__init__(face_detector=None, face_classifier=None, batch_len=32, verbose=False)
Construct a face processing pipeline composed of a face detector,

a face preprocessing strategy and a face classifier. The face preprocessing strategy is defined based on the classifier’s properties.

Parameters:
class inaFaceAnalyzer.inaFaceAnalyzer.VideoKeyframes(face_detector=None, face_classifier=None, batch_len=32, verbose=False)[source]

Bases: FaceAnalyzer

Face detection and analysis from video limited to video key frames https://en.wikipedia.org/wiki/Key_frame It allows to provide a video analysis summary in fast processing time, but with non uniform frame sampling rate

__call__(video_path)[source]

Pipeline function for face classification from videos, limited to key frames

Parameters:

video_path (string) – Path for input video.

Returns:

frame position, coordinates, predictions, decision function,labels…

Return type:

Dataframe with frame and face information

__init__(face_detector=None, face_classifier=None, batch_len=32, verbose=False)
Construct a face processing pipeline composed of a face detector,

a face preprocessing strategy and a face classifier. The face preprocessing strategy is defined based on the classifier’s properties.

Parameters:
class inaFaceAnalyzer.inaFaceAnalyzer.VideoTracking(detection_period, face_detector=None, face_classifier=None, batch_len=32, verbose=False)[source]

Bases: FaceAnalyzer

Video processing pipeline including face detection, tracking and classification Tracking is usually less costly than face detection (computation bottleneck) and allows to save computation time Classification decision functions and predictions are averaged for each tracked faces, allowing to obtain more robust analysis estimates

__call__(video_path, fps=None, offset=0)[source]

Pipeline function for face classification from videos with tracking

Parameters:

  • video_path – str Path to input video.

  • fps – float or None (default) amount of video frames to process per seconds if set to None, all frames are processed (costly)

  • offset – float (default: 0) Time in milliseconds to skip at the beginning of the video.

Returns:

frame position, coordinates, predictions, decision function,labels… faceid column allow to keep track of each unique face found predictions and decision functions with ‘_avg’ suffix are obtained through a smoothing procedure of decision functions for all faces with same faceid. Smoothed estimates are usually more robust than instantaneous ones

Return type:

Dataframe with frame and face information

__init__(detection_period, face_detector=None, face_classifier=None, batch_len=32, verbose=False)[source]

Constructor

Parameters:

  • detection_period (int the face detection algorithm (costly) will be used once every 'detection_period' analyzed frames. Ie: if set to 5, face detection will occur for 1/5 frames and the) –

    remaining 4/5 faces will be detected through a tracking procedure

    if set to 1: face detection will occur for each frame. Face tracking will also be used for each frames, since it will allow to group same faces under a person identifier

  • face_detector (instance of face_detector.FaceDetector or None, optional) – if None, LibFaceDetection is used. The default is None.

  • face_classifier (instance of face_classifier.FaceClassifier or None, optional) – if None, Resnet50FairFaceGRA is used (gender & age). The default is None.

  • verbose (boolean, optional) – If True, will display several usefull intermediate images and results. The default is False.