scoring

Running the scoring scripts:

0. Install Python requirements: pip install -r requirements.txt. You may
   want to run everything in a virtual environment:

   python -m virtualenv venv
   . ./venv/bin/activate
   pip install -r requirements.txt

   You’ll need to activate the virtual environment every time before you run
   scoresd.py.

   If you have trouble installing MySQL-python, you can install pymysql
   instead.

1. Make sure MySQL is up and running, with a db ‘scoring’ with access
   to the user ‘scoring’ with no password (and make sure MySQL is
   listening only on localhost or has TCP connections disabled).

   create database scoring;
   create user scoring;
   grant all privileges on scoring.* to scoring;

2. mysql -uscoring scoring < database.sql will (re)create the tables,
   discarding any existing data.

3. Ensure that the the target directory for html exists. By default, this is a
   subdirectory scoring of the main repository, changed via the
   scoring-local config option.

4. python scoresd.py will start a daemon to update the db continuously
   from the logfile and milestones. This process logs to a file named
   scoresd.log, by default in the same directory as the daemon.

5. To stop the daemon, you can run python scoresd.py --stop, which will
   ensure that all currently updating pages finish. (This may take a bit of
   time.)

Other scripts:

For a one-off scoring update, you can run python scbootstrap.py. This is
mostly identical to running the first update pass from scoresd.py.

Script options

Neither scoresd.py and scbootstrap.py will do a complete player page rebuild
unless they have to. To trigger this, pass either script -p. (A player page
rebuild will also be triggered if the players directory is empty.) This may
be necessary if scoresd.py does not shut down gracefully for some reason; it
usually takes 3-4 hours for the full scoring database though. This may be
necessary if the daemon isn’t shut down properly.

To run scoresd.py without daemonizing, pass it -n. This will not log to a
file, so is recommended only for testing. When run in this mode, the daemon
will respond to ctrl-c; the db state should be rolled back to the last commit
if this happens, and page generation may or may not succeed.

For more command line options, see python scoresd.py --help.

Configuration

Configuration is done via sources.yml. Main global options:

• scoring-local: the location for generated html files. If relative, this will
  be relative to the location where scoresd.py is run from.
  Default: scoring
• scoring-base-url: the base URL for where scoring-local is exposed on a
  server.
  Default: a file:// URL reconstructed from scoring-local. This is useful
  for testing, but you will need to set this appropriately for running a
  server. This path should include the full path to wherever scoring-local
  exists on the server (this is a change from previous versions).
• local-base: the location for log, lock, and stop files.
  Default: . (i.e. the directory that scoresd.py is run from.)
• rawdata-base: a location for wherefiles.
  Default: None (this disables the wherefile functionality)
• use-milestones: whether to read in milestone files. For the full scoring
  database this is no longer feasible, but it may be reasonable to do for
  smaller installations. This must be set.
• daemon-name: used for naming log, lock, and stop files. E.g.
  scoresd.log.
  Default: scoresd. (Change from previous versions: this was was effectively
  scoring; but wasn’t configurable without editing python.)
• game-restrictions: see below
  Default: None
• sources: a list of sources. This must be set.

Setting sources: This is a list of source descriptions. Each consists of the
following properties:

• name: the internal name of the source. Can be anything.
• canonical_name: the “official” name of the source; used both for aggregating
  multiple source entries, and indicating server names to the viewer. If unset,
  will just use name.
• server_url: the URL that the server is located at. Can be None.
• base: for remote servers, a URL base to look for logfiles at.
• local: for local servers, a path to look for logfiles at.
• dormant: set to True for remote servers that are no longer accessible;
  scoring will still use the cached logfiles if any, but no longer try to
  download.
• logfiles: a list of logfile paths using either local or base as a
  starting point.
• milestones: if the server is using milestones, a list of milestone paths.
• morgues: a list of morgue base paths. These allow rather complex filtering;
  see the official CAO sources.yml for examples.

Game restrictions and bans

In the sources yml, you can provide two kinds of bans, as properties under
game-restrictions:

• botnames: accounts associated with bots. Games from these accounts are
  tracked and stored in the database, but not shown in the realtime
  leaderboards.
• buggy: a list of game_keys that are buggy and should not be shown outside
  of player pages.

scoresd also has a facility for banning players. To use this, create a file
called banned_players.yml. This file should have a property banned, which
consists of a list of player names. This primarily affects reading now data:
if a player name is on this list, logfile entries with that player name will be
ignored entirely.

If you need to ban a player after logfile lines involving that player have been
read, add them to the list, and run python ./scoresd.py --run-bans. Be aware
that this executes DELETE queries on the database, so it is not undoable
without a full database rebuild. In addition, if the player appears in
tables that are limited to a certain number of slots, the system will not fill
in a new game into the blank spot created by the DELETE operations, and the
empty slot may be filled in by subsequent logfiles even if it should in
principle be filled be previous logfile lines. (This situation can also only
be corrected by a full database rebuild.)