- Text Mining
- SAS
Key challenges
Siemens Healthineers is shaping the future of healthcare with a wide range of products and innovative services. The leading medical technology company always focuses on the safety of patients and staff. This is ensured, among other things, by meticulously documenting and evaluating all service cases for medical equipment. As soon as an incident occurs that could endanger safety or patients, the laws of numerous countries require official notification. If an incident is not reported, this can have serious consequences for the manufacturer.
In order to further optimize and secure this process, Siemens Healthineers and our team of data experts have developed a procedure to automatically analyze and evaluate the free text fields of service notifications. This is intended to significantly minimize the effort required for the time-consuming manual review of non-escalated service notifications and to further increase quality.
Safety first
It is extremely rare for a service case to meet the criteria for notification to the authorities. The number of actually relevant events identified is negligible. And yet: safety is the top priority in medical technology, so all reports must be examined in detail. Respective state authorities carry audits to ensure that no service case is overlooked by companies. Siemens Healthineers has established a two-stage escalation process for this purpose, which initially involves the recording and analysis of a service case. The employees in the service centre or the service technicians on site are specifically trained to directly identify the relevance of a service notification with regard to a potential security problem. If such a case should occur, the information is immediately forwarded to a team of experts who carry out an appropriate internal examination. As soon as the suspicion is confirmed, the notification to the authorities is immediately made. This is a well-rehearsed process which fortunately only has to be carried out very rarely. And yet: the challenge remains of how to ensure the effectiveness of the escalation process and correct compliance with legal requirements.
Reliably automated instead of manually checked
Until now, Siemens Healthineers has relied on statistically representative random samples and their manual testing – a very complex and very tiring task for the experts. It is quite clear that the use of computer helped analysis methods. However, this is not simply a matter of evaluating existing data. The service cases are too individual to be described with predefined text modules. Therefore, the employees use free text fields to document the events. In a first approach, a keyword search was used to identify safety-relevant cases afterwards. This grid quickly proved to be much too coarse. The goal: the automated checking and evaluation of all completed service cases reliably and verifiably on the basis of text mining.
Our approach
Preparation of the data for text mining
First the text mining system had to be trained. By which contents can critical incidents be identified? In order to develop a model, our team of data experts first compiled the texts to be evaluated from various sources into a table in the SAS DI Studio and then cleaned them up.
Afterwards, all terms had to be eliminated that were only included in the service ticket by the employees after the escalation. Names and email addresses also have no relevance for escalation, so that these also had to be filtered out. After the texts were cleaned up, they were systematically analyzed in the SAS Enterprise Miner and different competing statistical algorithms were developed for the individual product groups. The application using training data then showed which algorithm best met the predefined evaluation criteria. In general, text mining provides a score as a result, which indicates how likely it is that a safety-relevant incident is involved. If the score of a ticket that has not already been escalated is above the threshold value determined for each product line, the ticket is checked again manually. In fact, the values of our data team scoring model corresponded to the experts’ assessment, so that the system passed the last critical dry run.
Benefits
Siemens Healthineers introduces text mining for the verification of all service tickets in the USA and Germany. All completed tickets are transferred daily to the central business warehouse, forwarded to the SAS Analytics platform and evaluated with the text mining system. The advantages of text mining are obvious:
- All tickets are checked, not just random samples
- Testing can be carried out much faster
- Significant reduction of manual effort
Of course, the project team built in a quality check for the text mining itself: All service tickets – even those that have been escalated – go through Text Mining. This checks whether the already known security-relevant tickets are correctly recognized by the algorithm. The team also attached great importance to the simple usability of the entire system. If necessary, the entire scoring process can be quickly adapted or extended during productive operation without in-depth SAS knowledge. All in all, the results of the text mining project for the Siemens Healthineers team are thoroughly positive: the text mining works fully automatically and achieves the required results. In addition, proof was provided that the manual escalation process reliably detects and forwards the security-relevant cases.
Thanks to our
Author
