Add modular Ansible-based regression test suite (REST API validation) (SZ-22)
rk@tigase.net opened 4 weeks ago

Goal

Introduce an Ansible-based regression test suite to validate the core REST APIs after each milestone, bugfix, or feature merge — ensuring nothing is accidentally broken.

This will serve as a lightweight and modular test harness that can grow with the project.


Motivation

We want to verify (in a repeatable, automated fashion) that:

  • Core entities (like roles and users) remain accessible
  • Basic operations like create/delete/list continue to work after changes
  • Future modules can be added easily, ideally one playbook per feature

Tasks

  •  Create top-level Ansible playbook: verify.yml
  •  Define roles:
    •  roles/api/verify_roles.yml: List and validate system roles
    •  roles/api/verify_users.yml: Fetch and validate known users (e.g., admin)
    •  roles/api/create_test_user.yml: Create a user rk with role FOO
    •  roles/api/delete_test_user.yml: Delete user rk and re-validate
  •  Wire up each role into the top-level playbook
  •  Make playbook fail cleanly if any call fails or yields unexpected response

Assumptions

  • Backend runs locally at http://localhost:8080
  • Uses the Ansible uri module to make REST calls
  • Authentication is either open (for dev) or passed via basic auth/token

Stretch Goals

  •  Integrate with a Git hook or CI step
  •  Add support for checking response bodies (e.g., role names, user count)
  •  Add support for future UI or gRPC tests (in a separate playbook)

Estimate

Initial setup (core roles + user CRUD): 2–3 hours

Adding additional playbooks for new features: 15–30 mins per feature

  • rk@tigase.net commented 4 weeks ago

    Ansible-based Regression Test Suite

    Create a lightweight, modular test harness using Ansible playbooks to verify system functionality after key milestones (feature completion, bugfixes, refactors).

    Goal

    Ensure that basic API endpoints behave correctly and catch regressions early by running a fixed suite of tests. The suite will initially cover core roles and user operations.

    Initial Test Modules

    • verify_roles.yml – Check that the expected roles (including ADMIN) are present.
    • verify_users.yml – Ensure that the default admin user exists.
    • create_test_user.yml – Create a test user (rk) with a sample role (FOO).
    • delete_test_user.yml – Delete the test user and confirm it no longer exists.
    • verify.yml – Top-level playbook that invokes the above in sequence.

    Modularity

    Each new feature (e.g., Projects, Issues, PullRequests) can add its own role-based module and plug into the verify.yml sequence. The system will be designed to fail fast on error and output clear diagnostics.

    Tools

    • Ansible: Preferred for modularity and integration later into CI/CD.
    • httpie: Preferred over curl for clarity, but we’ll fall back to curl if httpie is unavailable on target system.

    Estimated Time

    • Setup and core module implementation: 2.5–3.5 hours
    • Adding future modules: ~15–30 minutes each
  • rk@tigase.net changed state to 'In Progress' 4 weeks ago
    Previous Value Current Value
    Open
    In Progress
issue 1 of 1
Type
New Feature
Priority
Normal
Assignee
Version
none
Sprints
n/a
Customer
n/a
Issue Votes (0)
Watchers (3)
Reference
SZ-22
Please wait...
Page is in error, reload to recover