9.1 The Audit Process

9.1.1 Audit Scope

Audits are primarily focused on the four COUNTER Reports. The Standard Views of the COUNTER Reports will pass the audit provided the metrics match those in the COUNTER Reports, with the appropriate aggregation.

Each report provider MUST be audited on all the COUNTER Reports and Standard Views of the COUNTER Reports they are required to provide. Report providers who elect to offer additional COUNTER Reports (e.g. eJournal Host_Types that offer Item Reports) SHOULD have the additional COUNTER Reports and Standard Views of the COUNTER Reports audited. If the elective reports are not audited, this must be documented in the audit report.

Any exclusions to the audit scope MUST be agreed with the Project Director in writing prior to an audit commencing (tasha.mellins-cohen@counterusage.org).

9.1.2 Audit Scope For Third-Party Report Providers

Some publishers work with third parties to provide COUNTER reports. In these cases it is the third party report provider that is audited. Where such third parties are supporting multiple platforms, they must provide the auditor with a list of all platforms and the related COUNTER Reports and Standard Views. The auditor will then select a minimum of two and a maximum of five platforms from the list and audit accordingly, with the list to be approved by the COUNTER Project Director before the audit begins.

Where different technologies are used to generate reports for different platforms (e.g. data is harvested differently from one platform to the next), each technology MUST be audited separately.

9.1.3 Stages of the Audit Process

Audits follow a five-stage process.

  • Stage one - pre-flight preparation. Report providers SHOULD run all of their COUNTER reports (JSON and tabular) through the COUNTER Validation Tool. Report providers SHOULD also seek feedback from large report consumers (e.g. JUSP) in respect of issues they have identified with the reports. Report providers SHOULD fix any problems that are discovered and re-run the Validation Tool before proceeding to Stage two.

  • Stage two - audit initiation. Auditor and report provider agree on the scope of the audit, which MUST include all COUNTER Reports and Standard Views of COUNTER Reports required by the Code of Practice but SHOULD include all COUNTER Reports and Standard Views of COUNTER Reports offered by the report provider. COUNTER’s Project Director will adjudicate if the auditor and report provider cannot agree on the audit scope. Report providers MUST provide pre-flight documentation to the auditor, using the template provided in Appendix E. If Stage one has been followed, report providers MUST provide the Validation Tool results to the auditor.

  • Stage three - seeding, audit begins. In this stage the auditors perform actions on the audited platforms in line with Appendix E. Seeding takes a full calendar month.

  • Stage four - report reconciliation. The auditors reconcile the COUNTER reports generated by the report provider with the seeding actions they undertook in Stage three. The outputs from this stage may be an Interim Report or a Pass Report, which MUST be shared with the COUNTER Project Director. If an Interim Report is issued,

    • The report provider MUST fix the issues identified by the auditor within three months of the date of the interim report

    • The auditor will re-reconcile the COUNTER reports

  • Stage five - audit complete. Once a Pass Report is issued, COUNTER will update the Registry of COUNTER compliant platforms to reflect the updated audit status of all platforms covered by the audit (COUNTER Registry).

Note that if it is not possible to re-reconcile the COUNTER reports after a report provider has completed their fix process, the auditor will need to re-seed the audit. The Project Director SHOULD be notified when this occurs.