engineering / Maintra journal
Maximo regression testing with an AI agent
Plain-English Maximo tests run by an AI agent in a real browser, checked against REST and SQL, then replayed with no model at all. How maxqa works on MAS 9.2.
A patch is going into the test environment on Thursday. Someone has to prove that work orders still approve, purchase orders still receive, PMs still generate and the storeroom still issues. In most Maximo shops that someone opens a spreadsheet of test cases and clicks through them for two days.
The Selenium suite that was meant to do this has not run since the last upgrade. The login page changed, half the locators were generated control ids, and nobody on the team writes Java.
Maximo regression testing with an AI agent works when you split the job. A model reads a screen well and works out which button comes next. It also makes mistakes and reports them as success. So the agent drives the browser, deterministic checks against the database decide the verdict, and the agent is only called back when the recorded run breaks.
Why Maximo resists ordinary test scripts
Generic browser automation assumes a page behaves like a web form. Maximo’s classic UI does not, and most broken test suites I have seen failed on one of these:
| Maximo behaviour | What a generic script does | What it needs instead |
|---|---|---|
| MAS 9 wraps the classic UI in the manage shell, with the app inside an iframe | Searches the top page and finds nothing | Find the app frame, fall back to the main frame on 7.6 |
Control ids such as m6a7dfd2f_tfrow_[C:1]_txt-tb are generated | Records the id; the next build breaks it | Target by kind and label: the Asset field, the Labor table |
| Fields only react to real keystrokes | Sets the value programmatically; Maximo never sees it | Type key by key, commit with Tab or Enter |
| A long-poll request is always open | Waits for network idle forever, or not at all | Wait until Maximo’s own event queue is idle |
| Smart-fill: “matches a number of possible values” | Treats it as an error | Open the lookup and pick the exact match |
| Some success messages carry E codes, such as BMXAA3280E on a PO status change | Fails a passing test | Read the message key, not the severity letter |
IBM does publish a Maximo Test Automation Framework for validating Manage processes. It is a Java and Selenium script library for out-of-the-box flows, and it has to be adapted to each customer’s configuration and to each MAS release. That suits a team with Java developers. Most Maximo teams I work with have administrators and functional leads instead.
The underlying issue is that Maximo’s business rules surface through a UI event queue, so a test has to go through the screen to exercise them. That is the reason Maintra keeps asset and work rules in backend services behind one API: a test can call the same rule the screen calls.
A test written the way you would brief a tester
A maxqa test is a YAML file with four parts: setup data, plain-language steps, checks, and a recording that is filled in automatically.
name: Approve a new work order via Change Status
app: WOTRACK
tags: [smoke, workorder]
setup: # fresh data through REST on every run
- rest_create:
os: mxapiwodetail
data: {siteid: BEDFORD, description: "maxqa approve {{run_id}}", assetnum: "11430"}
save: {wonum: wonum}
steps: |
1. Find work order {{wonum}} in the list and open it.
2. Change its status to Approved (APPR) using the Change Status action.
expect: # checked by code, not by the model
- rest: {os: mxapiwodetail, where: 'wonum="{{wonum}}"', field: status, equals: APPR}
- sql: {query: "select status from workorder where wonum='{{wonum}}'", equals: APPR}
- no_error_messages: true # no BMXAA....E shown during the run
- message: BMXAA4591I # this message must have appearedEach run creates its own work order through the Maximo REST API, tagged with a run id. Tests never depend on demo data or on the previous run’s leftovers.
What the agent actually sees
A screenshot costs thousands of tokens and a small model reads it badly. maxqa’s driver hands the model a compact, numbered list of what a user can act on instead, built from Maximo’s own label conventions:
APP: Work Order Tracking TAB: Work Order
@e1 field "Work Order" = "1215" [required]
@e2 field "Work Order description" = ""
@e5 field "Asset" = "" [lookup=asset]
@e14 field "Site" = "BEDFORD" [readonly]
@e16 field "Work Type" = "" [lookup=worktype]Each element carries its kind, label, current value, readonly and required flags, its lookup type, and any field error. When a dialog is open, the list is scoped to the dialog. New BMXAA messages from the status bar are added on top.
The model replies with one JSON action per turn, such as typing 11430 into @e5 and pressing Tab. That keeps the loop identical across an Ollama model, the Claude or Codex command-line tools on a subscription, or an API key.
Where Maximo needs a specific sequence, the driver does it rather than the model.
The clearest example is lookup_select. It opens the Select Value dialog, widens Filter By to ALL and presses Refresh, because Filter By only applies on Refresh and Refresh wipes typed filters. Then it searches with an exact =value filter and clicks the row whose key matches.
A tester learns that order by getting it wrong. The tool now does it in one step.
Application metadata comes straight from the Manage database when a connection is available. The presentation XML gives every label to attribute binding, MAXATTRIBUTE gives types and required flags, and the synonym domains give valid statuses. The agent may run read-only SQL; writes are rejected.
The model never decides the verdict
A test passes only when the agent reports success and every deterministic check passes. The model cannot pass a test on its own say-so.
In the reference environment, a small local model once picked the wrong row in an asset lookup: 11400 instead of 11430. The service request saved and Maximo showed BMXAA4205I, “Record has been saved”. The REST check read assetnum back as 11400 and failed the test. The model flagged the mismatch itself that time, but the verdict did not depend on it noticing; a test that trusted the save message would have passed.
When a test fails, the report explains why in Maximo terms. maxqa reads MAXMESSAGES once from your database and turns each message the run hit into its business key, the values filled into it, and a fix hint.
In that Create Labor run, the model typed a new labor code and Maximo replied with BMXAA3075W, asking whether to create a person record. The model treated the question as an error and retyped the value until the loop guard stopped it. maxqa warns after three identical actions and aborts at five.
That principle of AI proposing and evidence deciding is the same one behind Maintra’s AI maintenance assistant, where proposed actions go through approval workflows before anything changes.
Record once, replay with no model
Every passing agent run is saved as semantic steps, never as DOM ids. This is the recording for receiving a purchase order:
replay:
- op: open_app
app: receipts
- op: type
target: {kind: filter, label: PO, table: POs}
text: ={{ponum}}
then: enter
- op: click
target: {kind: button, label: Actions, table: Material Receipts}
- op: click
target: {kind: option, label: Select Ordered Items}
- op: click
target: {kind: button, label: Select All Records, table: Ordered Items}
in_dialog: Select Ordered Items
- op: click
target: {kind: toolbar, label: Save Receipt}A person can read it, review it in a pull request and fix it by hand. Record numbers are stripped from table names and chosen values become variables, so the recording survives regenerated control ids and fresh data.
Three run modes cover the lifecycle:
| Mode | What runs | Model needed | Use it for |
|---|---|---|---|
| agent | The agent works through the plain-language steps | Yes | Writing a test, exploring a flow |
| replay | The recorded steps | No | Fast, repeatable regression |
| auto | Replay first; if it breaks or a check fails, the agent takes over and re-records | Only on a break | Nightly runs |
In the reference environment, a recording with a renamed action broke during replay. In auto mode the local model finished the test from the plain-language steps and saved a new recording.
These are the core flows on the MAS 9.2 reference environment, a Manage and DB2 install with IBM’s demo data, written by a local 8B model running on Ollama:
| Flow | App | Model calls to write it | Replay |
|---|---|---|---|
| Plan a work order with labor and material | WOTRACK | 15 | No model |
| Work order lifecycle, APPR to INPRG to COMP | WOTRACK | 14 | No model |
| Create and approve a purchase requisition | PR | 13 | No model |
| Create and approve a purchase order | PO | 15 | No model |
| Receive a purchase order | RECEIPTS | 59 with a larger model, then 8 locally | No model |
| Generate a work order from a PM | PM | 8 | No model |
| Issue an item to a work order | INVISSUE | 10 | No model |
The full suite of eleven tests replays unattended in about five minutes, twenty to thirty-five seconds a test. Those figures come from demo data. A heavily customised estate will have its own numbers, so treat them as a record of what the tool did there.
It gets better without retraining
No model weights change. The tool improves through a verified memory stored as plain YAML in the workspace:
- Lessons. After a troubled run, one reflection call distils general rules, such as setting Site before Asset on a service request. A lesson is shown to the agent for its app, and again the moment its BMXAA code appears on screen.
- Track record. A lesson starts as a candidate, becomes trusted after two passing runs, and is retired if it keeps not helping. A wrong lesson can cause a failure, never a false pass, because the checks still decide.
- Teacher and student. A local model is the student. When it fails, a stronger model on a subscription reflects on the failure, the student retries with the new lessons, and if that fails the teacher runs the test itself and compares the two runs.
- Promotion. Lessons that keep being used are flagged for a developer to build into the driver. Smart-fill handling started as a failure the agent kept hitting; now it is driver code and costs no tokens.
The Create Labor test followed this path. After the reflection lesson, plus a driver change that marks Maximo’s yes or no prompts as questions, the same 8B model passed in six steps. The recording now replays in about twenty seconds.
Knowing what is different about your Maximo
No two Maximo estates match the manual. maxqa estate scan reads the Manage database once, read-only, and builds a configuration profile alongside the message catalogue. The profile holds customer-added fields and objects, cloned apps, security restrictions, conditional UI, automation scripts and their launch points, workflows, escalations, organisation and site settings against IBM defaults, and autonumber prefixes.
At run time the agent gets at most fourteen notes for the app under test, scoped to the test user’s site. For SR in the reference environment, that is a single line: the Service Request workflow starts from the Route Workflow button. A tester would want to know that before writing the steps, and so does the agent.
The same profile powers maxqa estate diff, which compares two environments or one environment across an upgrade. Run the pack on both sides of a MAS upgrade, then diff the configuration. You get a list of what changed and a list of what still works.
Preparing release evidence with automation while a person approves production is how Maintra treats agent-assisted delivery too.
What maxqa is, and what it is not
maxqa is in closed beta. It is a regression tool for the pack a Maximo functional team signs off before a patch, an upgrade or a move to MAS: the business processes that must still work on day one.
| maxqa is | maxqa is not |
|---|---|
| A test tool for Maximo Manage that turns plain-English steps into replayable regression tests | An assistant clicking around production |
| Run on the tester’s machine against test and UAT environments | A hosted service that holds your Maximo credentials |
| Verified on every run by REST, SQL and message checks | A recorder whose pass depends on what a model saw |
| Model-agnostic: local Ollama, Claude or Codex subscriptions, Anthropic or OpenAI keys | Tied to one AI vendor |
| Readable YAML recordings that can be reviewed in a pull request | A black box nobody can audit |
The time saved comes from replay. Once a flow is recorded, the eleven-test core suite runs unattended in about five minutes with no model calls, and it can run every night. Each run leaves screenshots, the result of every check, an HTML report and JUnit XML for a CI pipeline. That is the evidence a test lead attaches to a sign-off, produced without anyone clicking.
The beta covers the classic Manage UI on MAS 9.2 with DB2. Validation on Maximo 7.6, role-based Work Centers, single sign-on, MAS SaaS estates without database access, and parallel runs are on the beta roadmap.
Before your next regression cycle
- List the flows you click through by hand Work order approval, PR to PO to receipt, PM generation and storeroom issue cover most packs. Write each one as three to six plain-English steps.
- Give every test its own data Create the record in setup through REST with a run id in the description. Never test against a work order someone else might touch.
- Write the check before the steps Decide which stored field proves the test passed, and assert it with REST or SQL. Add a message check for the BMXAA code you expect.
- Scan the estate Read your message catalogue and configuration once, so the failures your tests hit are explained in your system’s own terms.
- Replay nightly, heal on break Run in auto mode. Look at the runs that needed the agent; they show you where the environment changed.
Let the model drive the browser. Never let it mark the exam.