Testing

Tests protect the trading platform from regressions. Follow the test pyramid: put most coverage at the unit layer, integration tests in the middle, and a small, high-value end-to-end layer. Do not turn it into an “ice-cream cone” with slow, fragile end-to-end tests.

The Testing Handbook is NinjaTrader’s main testing guide. This page sets the standard. The handbook explains how to meet it for each stack.

Layers

Keep test data isolated and disposable. A bugfix must include a test that fails without the fix. Flaky tests are defects. Quarantine them with a ticket or delete them. Never rerun them until they pass. No automated suite writes to production.

Frameworks per surface

Make “unit” specific. Use the standard framework for each language in the stack.

Property-based testing

Example tests cover the cases you considered. Property-based tests cover input cases you did not consider. Use them for trade math, FIFO matching, FIX round-trips, and settlement boundaries. Use ScalaCheck on the JVM, fast-check for TypeScript, and Hypothesis for Python. The NT-NinjaTrader/testing-handbook ScalaCheck cookbook (pages/stack-cookbooks/backend-services/scala/property-scalacheck.md) is the official in-house example, and tradovate/servers already includes ScalaCheck.

import org.scalacheck.Prop.forAll

// FIFO matching conserves quantity: fills never create or destroy shares.
property("fifo conserves quantity") = forAll { (orders: List[Order]) =>
  val fills = matchEngine.run(orders)
  val buyQty  = orders.filter(_.isBuy).map(_.qty).sum
  val sellQty = orders.filter(_.isSell).map(_.qty).sum
  fills.map(_.qty).sum == buyQty.min(sellQty)
}

import fc from "fast-check";

// FIFO matching conserves quantity: fills never create or destroy shares.
it("fifo conserves quantity", () => {
  fc.assert(
    fc.property(fc.array(orderArb), (orders) => {
      const fills = matchEngine.run(orders);
      const buyQty = sum(orders.filter((o) => o.isBuy).map((o) => o.qty));
      const sellQty = sum(orders.filter((o) => o.isSell).map((o) => o.qty));
      return sum(fills.map((f) => f.qty)) === Math.min(buyQty, sellQty);
    }),
  );
});

from hypothesis import given, strategies as st

# FIFO matching conserves quantity: fills never create or destroy shares.
@given(st.lists(orders()))
def test_fifo_conserves_quantity(order_list):
    fills = match_engine.run(order_list)
    buy_qty = sum(o.qty for o in order_list if o.is_buy)
    sell_qty = sum(o.qty for o in order_list if o.is_sell)
    assert sum(f.qty for f in fills) == min(buy_qty, sell_qty)

The trading protocol uses more than one wire format. Add golden or snapshot tests and parser fuzzing for FIX serialization round-trips and message parsers.

Integration testing

“Real dependencies in containers” means Testcontainers: start a real Postgres, MySQL, or Kafka instead of mocking the seam under test.

class LedgerSpec extends AnyFlatSpec with Matchers with ForAllTestContainer {
  override val container = PostgreSQLContainer()

  "ledger" should "persist a posting" in {
    val repo = new LedgerRepo(container.jdbcUrl)
    repo.post(entry)
    repo.balance(acct) shouldBe entry.amount
  }
}

Contract testing

End-to-end tests for every service pair are fragile and slow. Use consumer-driven contract tests with Pact at service seams. This lets the Scala, TypeScript, and Node services change independently. Verify contracts against the domain model in tradovate/master-scheme.

Determinism

Flaky tests are defects. Remove their causes:

• Inject the clock. Do not use System.now or DateTime.now in code under test.
• Do not use Thread.sleep or arbitrary waits. Wait for a condition, not a set time.
• Seed all randomness so failures can be reproduced.
• Fix the timezone and locale.

Coverage and mutation testing

Coverage shows whether lines ran. It does not show whether tests would catch a defect. Gate on diff coverage for changed lines instead of a global percentage, and enforce it through quality gates. Measure suite strength with mutation testing, using PIT for the JVM and Stryker for TypeScript. Run it periodically, not as a per-PR gate.

Load testing

State a latency or throughput budget wherever one exists, and fail the build when it is exceeded. Use Gatling (Scala-native) or k6. Connect load budgets to the SLOs and golden signals in observability.

export const options = {
  thresholds: {
    http_req_duration: ['p(95)<250'], // 95th percentile under 250ms
    http_req_failed:   ['rate<0.01'], // under 1% errors
  },
};
// k6 exits non-zero when a threshold is breached, so CI fails.

Security testing

Run SAST for each pull request, DAST against web and mobile endpoints, and a third-party penetration test every year. The security standard owns these controls and the OWASP baselines behind them.

Accessibility testing

Automate the testable half of WCAG 2.2 with axe-core or Lighthouse for web, and Flutter’s accessibility APIs for mobile. Automation is required but not enough. Also do a manual screen-reader check.

import { axe } from 'vitest-axe';

test('checkout page has no a11y violations', async () => {
  const { container } = render(<Checkout />);
  expect(await axe(container)).toHaveNoViolations();
});

Related standards

• Quality gates, how test layers and coverage are enforced as required checks.
• CI/CD, where suites run, shard, and fail the build on thresholds.
• Security, the SAST, DAST, and penetration-testing controls.
• Observability, the SLOs and latency budgets that load tests defend.