Tech to medically check pilots had a dozen major disadvantages - CAA papers

Aviation officials have found big problems with software they tried to impose on doctors doing medical check-ups on pilots and air traffic controllers.

Doctors rebelled against the new IT system last month, citing fears it would make it harder to accurately report on pilots' medical conditions.

The Civil Aviation Authority (CAA) told RNZ there had never been any safety concerns, but its own analysts agreed with doctors' worries.

They "believe they could miss vital historical information relevant to a current examination", according to internal papers newly released to RNZ under the OIA.

"The analysis supports [doctors'] concerns."

The papers show that when the CAA finally did an analysis, it revealed a dozen major disadvantages of the new tech, compared to existing systems.

This included "medical history documents not easily searchable", "no drill-down reports" and not being able to get tests - such as bloods - done before a pilot got a check-up.

It "would have saved a lot of effort (and some money) had they performed that level of analysis at the start", said Dr Greg van der Hulst, one of about 50 doctors nationwide qualified to do the aviation medicals.

He led the public rebellion by the Aerospace Medical Society (AMSANZ) that [https://www.rnz.co.nz/news/national/513420/civil-aviation-backtracks-over-technology-for-medical-checks-on-pilots-air-traffic-controllers forced the CAA into a u-turn in April.

"It appears the medical module part of the EMPIC project was poorly informed from the start due to the lack of early engagement with the medical examiners who work in this area every day," van der Hulst told RNZ this week.

CAA said in a statement late on Tuesday it had not been able to resolve the search problem, but that this "does not present a safety concern, but it does raise an inefficiency that can interfere with the process".

Asked why it did not do the analysis two years ago, it said its' focus had been on how to record the outcomes of check-ups, and it was "not focused on providing new software" to the doctors.

EMPIC was chosen based on all its regulatory requirements - not just medical, project sponsor and deputy chief executive Mike Hill said.

A history of issues

The OIA papers showed the project went off track from the start.

One reason CAA chose to buy the system for a couple of million dollars was it was meant to be off-the-shelf and good to go. It did not run an open tender when it chose EMPIC.

Instead, the project had stuttered along.

It started in November 2021, but was paused four months later for EMPIC to do an upgrade "to better meet" what CAA needed. Six months later it required more "enhancements".

Even after this, the medical reports it was able to produce did not fit with New Zealand's approach to medical check-ups, which was more flexible than in many countries.

The project restarted again in May 2023, involving extra costs for work on customising the reports.

By this time, rival home-grown system SkyCert had been raising concerns for years, including protesting to the transport minister that the CAA was robbing local business, emails showed.

SkyCert currently handled 60 percent of the 6000 pilot and air traffic controller checkups a year, including all 1600 for Air New Zealand.

The belated internal analysis, across five pages in the OIA, compared EMPIC to SkyCert, which came off better.

At one stage, the CAA told doctors that SkyCert's system did not send through suspension notices. This would be bad - but was wrong. CAA later apologised to SkyCert, emails show.

By last October, the CAA was still wedded to EMPIC, even as it was having to "redefine the questions for the online medical application form to meet the known constraints of the software".

"This provided some improvements, but still had a level of compromise that was unacceptable to CAA," its internal report said.

In December, the CAA decided to bring doctors on board, forming a "user group" to "provide input into the design".

Doctors have told RNZ this came too late.

The newly-released internal report was written to show why the CAA had to give up on the forced introduction of the medical module. The module was part of a bigger EMPIC aviation regulatory system that the CAA was still introducing, and it had pointed out that it was widely used by aviation regulators world-wide.

EMPIC had been approached for comment.

Listening to feedback, or wasting money?

The authority told RNZ its u-turn in April showed it had listened to pilots' feedback.

"The CAA has a responsibility both to consult and to explore opportunities that maximise the potential of any investment, which is what has happened in this case," it said.

But EMPIC was not being maximised, it was being trimmed back. The CAA now intended to use it internally only, although it still aimed to get the tech to the point that there can be a universal rollout to doctors.

SkyCert director Chris Lapish said that was a waste.

"Management at the CAA are determined not to stop wasting money trying to force on the industry a poor and expensive option," Lapish told RNZ this week.

The OIA papers show the April u-turn was also driven by "strong resistance" from doctors "which is almost certain to result in low adoption", and the CAA worried about media picking up the story. RNZ first reported on it in March.

"The project's view is that to proceed with the current implementation would create greater harm to relationships with [doctors], and could damage CAA's reputation with the NZ public."

One driver of the entire project was to "reduce risk" from the old tech, according to official documents.

CAA's leadership told RNZ in April that "there are not, and have never been, any concerns about safety with regard to processing medical certificates".

But fresh problems have cropped up over how the rejigged system would work, now that doctors would not use EMPIC, while CAA would.

The papers showed that two options being looked at that covered 10 priorities - and seven of those were listed in bright red and marked as "does not achieve".

"Regulatory effectiveness: whether the solution improves the information being assessed by the regulatory decision maker" scored "does not achieve".

Either option created multiple "gaps" that required workarounds.

One workaround missed out "a critical requirement", and another "cannot generate a medical certificate".

There were "too many workarounds within system to record accurately what has occurred".

The work had "raised new questions on the use of additional data fields and the ability to report on those", the CAA paper said.

"These questions have been raised with EMPIC. EMPIC had also been asked to advise what their plans are for modernising the medical module to address the concerns of ... future use in New Zealand."

Hill said the option the authority was going with could generate the medical certificate and met the "critical requirement".

"There is no need to improve the workarounds, since they match the current processes from our existing system and there are no safety concerns.

"In the future, we will look to develop the medical module further, with the vendor, to enable medical examiners to manage more of the process within our system."