lookout.style.reporter

General utilities to generate performance reports for analyzers.

Module Contents

class lookout.style.reporter.Reporter(config:Optional[dict]=None, bblfsh:Optional[str]=None, database:Optional[str]=None, fs:Optional[str]=None, checkpoint_dir:Optional[str]=None, force:bool=False)

Base class to create performance reports for the analyzer.

To create a reporter for your Analyzer you should make two steps. 1. Inherit SpyAnalyzer from an Analyzer you want to evaluate. SpyAnalyzer’s analyze function should be overridden to return all the information you need for the following evaluation of the Comment-s. Refer to TyposAnalyzerSpy as an example. 2. Inherit MyReporter from this Reporter class. Set created SpyAnalyzer to inspected_analyzer_type attribute. You should have a dataset that you feed to Reporter.run(). The dataset rows are passed to _trigger_review_event to trigger your analyzer’s analyze(). The result is passed to _generate_reports(). If you need to summarize your reports, override _finalize method.

If you want to create several reports (e.g. separate train and test reports) you should override both get_report_names() and _generate_reports().

_log

inspected_analyzer_type

__enter__(self)

__exit__(self, exc_type=None, exc_val=None, exc_tb=None)

run(self, dataset:Sequence[Dict[str, Any]])

Run report generation.

Parameters:dataset – The dataset for the report generation. The format is a list of data rows. The row is a Dictionary with a mapping from the column name to its content. Returns:Iterator through generated reports. Each Generated report is extended with the corresponding row data from the dataset.

classmethod get_report_names(cls)

Get all available report names.

Returns:Tuple with report names.

_generate_reports(self, dataset_row:Dict[str, Any], fixes:Sequence[NamedTuple])

Generate reports for a dataset row.

Parameters:

• dataset_row – Dataset row which triggered the analyze method of the analyzer.
• fixes – List of data provided by the analyze method of spied analyzer.

Returns:

Dictionary with report names as keys and report string as values.

_trigger_review_event(self, dataset_row:Dict[str, Any])

Trigger review event and convert provided comments to an internal representation.

It is required to call Reporter._analyzer_context_manager.review() in this function with arguments you need and convert provided comments to a Sequence of NamedTuple-s for the report generation.

Parameters:dataset_row – Dataset row with information required to run analyzer_context_manager.review(). Returns:Sequence of data extracted from comments to generate report.

_finalize(self, reports:Iterable[Dict[str, str]])

Extend or summarize the generated reports.

The function does not change the reports by default.

Parameters:reports – Iterable with generated reports. Returns:New finalized reports.

static _get_package_version()

Return lookout-style package version or “local” if it is a git repository.

static _get_commit()

Return current head commit hash if you run inside git repository.

static _get_row_repr(dataset_row:Dict[str, Any])

Convert dataset row to its representation for logging purposes.