Beacon api compliance suite

Validation:

• The beacon OpenAPI schema (GA4GH)
• The JSON schemas (CSCfi)

Data:

• Mainly from 1000 Genomes Projet
• 11 lines (17 variants) from chromosome 22, Phase 3
• 2 structural variants (BND, for version 1.1.0 )

The test suite...

• validates the structure and content type of the responses

• tests

  • frequency, variantCount, callCount, sampleCount
  • searching by alternateBases and by variantType
  • variant types (DEL, INS, SNP, BND)
  • span queries (startMin/startMax)
  • errors (queries without assemblyId etc)

Testing a beacon:

• Import the data (a vcf-file)

• Requirements:

  • python3
  • pip3
  • pip3 install -r requirements.txt

• Run the test suite:

  python3 beacon-api-tester.py --host "http://mybeacon.com"

Output:

Output, validation failure:

• "organization": null

Output, validation failure (only from the JSON schemas):

• assemblyId: ""

Output, validation failure (only from the OpenAPI spec):

• "organization": {"info": {"orgInfo": "..."}}

Output, bad answer:

Output, bad counts:

Schema validation:

• will be automatically downloaded from GA4GH's / CSCfi's Github.

• the queries are also validated (may help finding errors in the specifications).

Not tested:

• other values than HIT for includeDatasetResponses

• other variantTypes than DEL, INS, SNP, BND

Future:

• add more test data?
• other types of checks?
• closer integration to the Beacon itself?
• all contributions are welcome!

    https://github.com/NBISweden/beacon-api-tests