Jenkins Jobs

Performance Analysis (PA)

CSIT PA runs performance analysis including anomaly detection as described above. PA is defined as follows:

  1. PA job triggers:
    1. By PT jobs at their completion.
    2. On-demand gerrit triggered.
  2. Download and parse archived historical data and the new data:
    1. Download RF output.xml files from latest PT job and compressed archived data from nexus.
    2. Parse out the data filtering test cases listed in PA specification (part of CSIT PAL specification file).
  3. Re-calculate new groups and their averages.
  4. Evaluate new test data:
    1. If the existing group is prolonged => Result = Pass, Reason = Normal.
    2. If a new group is detected with lower average => Result = Fail, Reason = Regression.
    3. If a new group is detected with higher average => Result = Pass, Reason = Progression.
  5. Generate and publish results
    1. Relay evaluation result to job result.
    2. Generate a new set of trend summary dashboard, list of failed tests and graphs.
    3. Publish trend dashboard and graphs in html format on https://docs.fd.io/.
    4. Generate an alerting email. This email is sent by Jenkins to csit-report@lists.fd.io