In the wake of openSUSE/osc@f1c3156 removing the deprecated submit/target,
the special condition to limit Factory to "staging era" requests should be
updated to look for action/target.
Now that the bug related to openSUSE/open-build-service#5571 has been fixed
the inverted behavior of exclude_target_projects no longer works. As such
the argument should be removed and only way to get the desired behavior
is to override include_request_from_project which includes request sourced
from specific project. For the purposes of metrics only interested in
requests targeting a specific project.
As the remote config is no longer optional for SLE and is utilized by
openSUSE to the point were it is dangerous not to load the remote config
it should be required. Currently only certain users call apply_remote()
while this will make it built-in during construction and thus makes the
usage consistent and no longer require StagingAPI.
This reverts commit 3fa61c8b8780b8de14f821fd0768bdd0b82ce6da.
Remove this garbage from repository as it is not a toilet. This creates
corrupt records and prevents deployment to metrics.o.o.
After change from dropping entire database to individual measurements this
was overlooked and instead new points are simply written each time which
results in duplicates.
This allows for non-ephemeral queries to be cached indeinitely in the
future (like source queries that include the revision). For development
one may use --heavy-cache to be able to quickly iterate.
Savings of around 400MB per 10,000 requests. Using a named tuple was the
original approach for this reason, but the influxdb interface requires
dict()s and it seemed silly to spend time converting them. Additionally,
influxdb client already does batching.
Unfortunately, with the amount of data processed for Factory that will
continue to grow this approach is necessary. The dict() final structures
are buffered up to ~1000 before being written and released. Another
benefit of the batching is that influxdb does not allocate memory for the
entire incoming batch.
This prevents the memory required to process to be enough to load all
parsed request element trees at once. Instead only one page of requests
is loaded at a time and the memory freed after processed. The end result
is the memory consumption reduced by just over 20% (current Factory drops
by around 2.5GB).
