pyvoa.geopd_builder

Assembly of a parsed database and its geography.

GPDBuilder joins the DataFrame produced by pyvoa.jsondb_parser to the matching geography, applies the statistical options a query asks for – daily and weekly differences, smoothing, cumulative sums, normalisation by population – and hands back one GeoDataFrame together with the visualisation dispatcher built for it.

Project : pyvoa Authors : Tristan Beau, Julien Browaeys, Olivier Dadoun Copyright ©pyvoa_org License : see the joint LICENSE file https://pyvoa.org/

class pyvoa.geopd_builder.GPDBuilder(db_name=None, parse=True)

Bases: object

One database, joined to its geography and ready to be queried.

Built from the name of a database: parses it through pyvoa.jsondb_parser, attaches the geometry of the locations it covers, and keeps the result as a single GeoDataFrame. get_stats() then applies to it the statistical options a query asks for – daily and weekly differences, smoothing, cumulative sums, normalisation by population.

front builds one of these per database selected with setwhom(); there is no reason to build one directly.

__init__(db_name=None, parse=True)

Build the GeoDataFrame of one database.

Calls the parser for the data, the geography for the locations, and the visualisation dispatcher for the charts, so that a GPDBuilder is usable as soon as it is built.

Parameters:

• db_name (str) – The database to load. With None, nothing is loaded.

• parse (bool) – True, the default, parses the database at once. False leaves it unparsed, for front.setwhom(reload=False), which reads the data back from the pickles instead: the parser is then only built if something asks for it, through get_parserdb().

getwheregeometrydescription()

Return the geometry of each location, as a ‘where’/’geometry’ frame.

gettypeofgeometry()

Return the geography object backing this database.

Returns:

• A GeoManager for a world-wide database, a GeoCountry otherwise.

• front reads its type to decide how to draw a map.

get_available_keywords()

Return available from the jsondb_parser.

factory(reload=True)

Provide the data, the geometry and the charts of the database.

The pickles of a database are written and read here, and nowhere else. With reload=True, splits the table the parser built into its data and its geometry and saves both as pickles in the pyvoa cache – the data with the list listwhich() gives and both answers of listwhere(), the geometry with the description of the locations. With reload=False, reads them back instead, without parsing: build the GPDBuilder with parse=False for that. Either way, builds the AllVisu that draws them, and keeps the two lists for getsavedlists().

Parameters:

reload (bool) – True, the default, parses and saves; False reads the saved pickles.

Returns:

(the data, without its geometry; one ‘where’/’geometry’ row per location; the AllVisu).

Return type:

tuple

Raises:

PyvoaError – With reload=False, if the database was never saved on this machine.

getsavedlists()

Return the variables and the locations saved with the data.

What listwhich() and listwhere() gave when the database was parsed, as factory() saved or read them back: front answers from them when it did not reload.

Returns:

‘listwhich’, a list; ‘listwhere’, both answers of listwhere() keyed on its flag – or a single list, from a pickle saved before both were kept.

Return type:

dict

getpklname(geoordata='geo')

Return the name of the pickle the data or the geometry was saved to.

Parameters:

geoordata (str) – ‘geo’, the default, for the geometry pickle, or ‘data’ for the data one.

Raises:

PyvoaError – If geoordata is neither.

getdatabase()

Return the whole database, as parsed, for every location and variable.

The table get() selects from, before any selection: this is the raw material, not a query. Its size is reported through info().

Returns:

Every variable of the current database, for every location and date.

Return type:

pandas.DataFrame

static split_data_geo(mypyvoageopd)

Split a geo DataFrame into its data and its geometry.

The geometry repeats on every date, which is wasteful to carry around and to pickle. Splitting it out leaves one row per location.

Parameters:

mypyvoageopd (gpd.GeoDataFrame) – the full frame.

Returns:

(data without the geometry column, one ‘where’/’geometry’ row per location).

Return type:

tuple

setvisu(db_name, wheregeometrydescription)

Build the AllVisu that draws the charts of this database.

Parameters:

• db_name (str) – The database the charts describe.

• wheregeometrydescription (gpd.GeoDataFrame) – One ‘where’/’geometry’ row per location.

getvisu()

Return the instance of Display initialized by factory.

setgeo(geo)

Set the geography object used to resolve locations.

getgeo()

Return the geography object used to resolve locations.

get_parserdb()

Return the DataParser holding the parsed database.

A builder made with parse=False parses the database here, the first time something needs the parser: the full table, the variable definitions and urls, or the description of the database.

get_available_GPDBuilder()

Return all the available Covid19 GPDBuilder.

subregions_deployed(listloc, typeloc='subregion')

Expand a list of locations into the individual places it covers.

A region name is replaced by the subregions it contains, so that a selection such as ‘Occitanie’ becomes the departements below it. Names already at the requested level are kept as they are.

Parameters:

• listloc (list) – the locations to expand.

• typeloc (str) – the level to expand to, ‘subregion’ (default) or ‘region’.

Returns:

the flattened list of locations.

Return type:

list

Raises:

PyvoaError – if an entry is neither a region nor a subregion of this database, or if a subregion is asked for at region granularity.

listwhich(dbname=None)

List the variables a database offers.

Parameters:

dbname (str, optional) – The database to ask about. Defaults to the one selected with setwhom().

Returns:

The values ‘which’ accepts, sorted alphabetically. Beware that the default of ‘which’ is not necessarily this first entry: the death count comes first when the database has one, see tools.prioritize_keyword.

Return type:

list of str

Raises:

PyvoaError – If no database was named and none has been selected.

listwhere(cluster_and_not=False)

List the locations the current database can be asked for.

What a location is depends on the granularity of the database: the countries of a world-wide or European one, the regions or the subregions of a national one. Clusters – the names standing for a group of locations, such as a continent, ‘European Union’ or ‘World’ – are listed alongside them.

Parameters:

cluster_and_not (bool) – If True, return the individual locations and the clusters – for a world-wide database, the countries under both their ISO3 code and their name. If False, the default here (front.listwhere defaults to True), return the clusters only.

Returns:

The locations, sorted. A database covering a single country is the exception: it returns that country’s ISO3 code alone, whatever this flag says, since there is nothing to choose from.

Return type:

list of str or str

Raises:

PyvoaError – If no database has been selected, if the selection is a table of your own (‘in-house data’, whose locations are yours to know), or if the granularity of the database is not one pyvoa knows.

whereclustered(**kwargs)

Handle the name and the geometry of a cluster of locations.

Returns a pandas in which the locations of a cluster have been collapsed: their names summed, their values summed or averaged depending on the variable, and their geometries merged. Names are compared upper-cased, so the comparison is case insensitive.

get_stats(**kwargs)

Return the keyword arguments filled in from the input arguments.

Tests every argument and its values, and applies the options asked for, in particular ‘nonneg’, which redistributes values so as to remove the negative ones.

The ‘when’ range cuts the database and a frame passed as ‘input’ alike. A single date is only selected at the very end, once ‘daily’ and ‘weekly’ have been computed, so that it keeps a day before it to be differenced against. ‘normalize:…’ writes its rate to a column of its own, ‘<which> normalize:…’.

normbypop(pandy, val2norm, bypop)

Return a pandas with a normalisation column added by population.

Can normalise by ‘100’, ‘1k’, ‘10k’, ‘100k’ or ‘1M’, naming the new variable accordingly.

saveoutput(**kwargs)

Save a pyvoa pandas as an output file.

Parameters:

• pandas (pd.DataFrame) – The pyvoa pandas to save. Mandatory.

• saveformat (str) – ‘excel’ or ‘csv’. Default is ‘excel’.

• savename (str) – The file name, without its extension. Default is ‘pyvoa_out’, hence pyvoa_out.xlsx or pyvoa_out.csv.

Notes

The ‘date’ column is rewritten as dd/mm/yyyy strings in place, on the frame given rather than on a copy of it.

smooth_cases(cases)

Smooth a daily series with a centred gaussian window.

Uses a seven-day gaussian window and drops everything up to the last zero, so that the series starts where the epidemic is actually under way. Used by the Rt estimation below.

Parameters:

cases (pd.Series) – the daily counts.

Returns:

the smoothed counts.

Return type:

pd.Series

get_posteriors(sr, window=7, min_periods=1)

Estimate the posterior distribution of the reproduction number Rt.

Bayesian day-by-day update over a grid of possible Rt values, with a gamma prior and a Poisson likelihood whose rate follows from the previous day’s count and a serial interval of seven days. Adapted from the kaggle notebook credited above the method.

Parameters:

• sr (pd.Series) – the smoothed daily counts.

• window (int) – the smoothing window, kept for signature compatibility.

• min_periods (int) – likewise.

Returns:

(posteriors over the Rt grid for each day, the log likelihood of the series).

Return type:

tuple