Working blind
Another challenge Vase identifies is what she calls “pseudo-data work.” Healthcare workers spend significant time correcting and validating the system’s suggestions to improve the algorithm, but they have no way of knowing whether their efforts have any effect.
“The problem is that they can’t see whether their corrections actually make a difference. The system’s statistical feedback isn’t available to them, so they risk working entirely in the dark,” she says.
In one extreme case, an entire department’s corrections were never forwarded to the vendor for a full year due to a technical error - and no one noticed.
Extra work for doctors
Automation of documentation has also led to a range of unanticipated practical tasks falling on doctors.
“Previously, medical secretaries scheduled X‑rays and other tests as part of transcribing the doctor’s notes. Now that doctors are responsible for documentation themselves, these tasks shift to them as well,” says Vase.
In some cases, this leads to bizarre workflows where doctors print out records and mark them up with a special code so that secretaries can later schedule X-rays or other tests.
“It’s yet another form of extra work that illustrates how AI doesn’t simply automate isolated tasks; it reshapes the entire division of labour. If systems aren’t designed with real‑world practices in mind, new types of extra work inevitably arise,” she adds.
AI must adapt to hospital workflows - not the other way around
According to Vase, greater transparency and support are needed when AI systems are introduced in healthcare. Otherwise, it becomes difficult for staff to understand - let alone engage with - the technology.
“Speech recognition isn’t just a tool that lightens the documentation burden. It fundamentally affects how staff work and creates informal collaborations where doctors and other healthcare professionals take on new roles and spend more time on tasks far removed from their core responsibilities. That also means that promised financial savings are sometimes far smaller than expected,” Vase says, adding:
“When AI technologies such as speech recognition are implemented in healthcare, it is crucial that they are adapted to existing workflows - not the reverse. The technology should serve the staff, who should not be forced to adapt to the system’s requirements, especially when no extra time is allocated for these tasks. That places demands on how these systems are fundamentally designed.”