Objective: The objective of this study is to implement and evaluate different personal health identifying information (PHI) substitution strategies in order to quantify their privacy preserving benefits. Background and Significance: Privacy considerations and legal mandates create a scarcity of clinical text for machine learning model training and evaluation. Synthetic generation of clinical notes has shown promise but more research is needed. An alternative approach is to use real notes and remove or replace only personal identifying information (PII), including PHI, through a process called de-identification. However, existing de-identification software may not replace identified PHI with surrogate text, making it difficult to train machine learning algorithms effectively. This challenge may lead to PHI data leaks if the de-identification process is skipped. Methods: In this study, we implement and assess three different "Hiding in Plain Sight" (HIPS) strategies for PHI replacement: a standard Consistent replacement strategy, a Random replacement strategy, and a novel Markov model-based strategy. We evaluate the privacy preserving benefits of these strategies on synthetic PHI distributions and real clinical corpora from two different institutions using false negative error rates (FNER). Results: Using FNER ranging from 0.1% to 5%, we observe that the Markov chain strategy significantly reduces PHI leakage compared to the Consistent strategy on a diverse set of notes from the University of Alabama at Birmingham (UAB). The Markov chain strategy also outperforms the Consistent and Random strategies on a MIMIC corpus of discharge summaries as well as on various synthetic clinical PHI distributions. Specifically, at 0.1% FNER document-level PHI leakage could be reduced from 27.1% to 0.1%, while at 5% FNER it could be reduced from 94.2% to 57.7%. Discussion: Our findings demonstrate that the Markov chain surrogate generation strategy substantially reduces the risk of inadvertent PHI release across different assumed PHI FNERs. We also release our implementation called "BRATsynthetic" on Github for the clinical informatics community to use. Conclusion: The Markov chain replacement strategy allows for the release of larger de-identified corpora at the same risk level compared to using a consistent HIPS strategy.
- Objective: Implement and evaluate different personal health identifying information (PHI) substitution strategies to quantify privacy preserving benefits.
- Background and Significance:
- Privacy considerations and legal mandates limit clinical text availability for machine learning model training.
- Synthetic generation of clinical notes has potential, but more research is needed.
- De-identification is an alternative approach, but existing software may not effectively replace identified PHI with surrogate text.
- Methods:
- Implement and assess three "Hiding in Plain Sight" (HIPS) strategies for PHI replacement: Consistent, Random, and Markov model-based.
- Evaluate privacy preserving benefits using false negative error rates (FNER).
- Results:
- Markov chain strategy significantly reduces PHI leakage compared to Consistent strategy on diverse set of notes from University of Alabama at Birmingham (UAB).
- Markov chain strategy outperforms Consistent and Random strategies on MIMIC corpus of discharge summaries and synthetic clinical PHI distributions.
- Document-level PHI leakage reduced from 27.1% to 0.1% at 0.1% FNER, and from 94.2% to 57.7% at 5% FNER using Markov chain strategy.
- Discussion:
- Markov chain surrogate generation strategy substantially reduces risk of inadvertent PHI release across different assumed FNERs.
- Implementation called "BRATsynthetic" released on Github for clinical informatics community use.
- Conclusion: Markov chain replacement strategy allows for release of larger de-identified corpora at same risk level compared to consistent HIPS strategy.
This is a very complicated text that talks about ways to protect people's personal health information when using it for research. They tried different strategies to replace the personal information with other words, but some strategies worked better than others. One strategy called "Markov chain" was the best at protecting the personal information. They tested this strategy on different sets of notes and it reduced the risk of releasing personal information a lot. They also made a program called "BRATsynthetic" that other people can use to protect personal information too."
Definitions- Personal health identifying information (PHI): Information about a person's health that can be used to identify them.
- Privacy: Keeping something private or secret, not letting other people see or know about it.
- De-identification: Changing or removing personal information from something so that it cannot be linked back to a specific person.
- Synthetic: Made or created artificially, not real or natural.
- Surrogate: Something that takes the place of something else, in this case, replacing personal health information with other words.
- Leakage: When something escapes or gets out unintentionally. In this case, it means when personal health information is accidentally released.
Privacy Preserving PHI Substitution Strategies: A Quantitative Evaluation
The privacy of personal health information (PHI) is an important consideration for medical research. In order to protect patient confidentiality, many organizations are required by law to de-identify PHI before releasing data sets for machine learning model training and evaluation. However, existing de-identification software may not replace identified PHI with surrogate text, making it difficult to train machine learning algorithms effectively. This challenge may lead to PHI data leaks if the de-identification process is skipped.
In this study, researchers implemented and assessed three different “Hiding in Plain Sight” (HIPS) strategies for PHI replacement: a standard Consistent replacement strategy, a Random replacement strategy, and a novel Markov model-based strategy. The objective was to quantify the privacy preserving benefits of these strategies on synthetic PHI distributions and real clinical corpora from two different institutions using false negative error rates (FNER).
Background and Significance
The scarcity of clinical text available for machine learning model training due to privacy considerations has led researchers to explore alternative approaches such as synthetic generation of clinical notes or removal/replacement of personal identifying information (PII), including PHI through a process called de-identification. However, existing de-identification software may not replace identified PHI with surrogate text, making it difficult to train machine learning algorithms effectively without risking inadvertent release of confidential information.
This study aimed at quantifying the privacy preserving benefits of HIPS strategies on various datasets in order to reduce the risk of inadvertent release of confidential information while still allowing larger datasets with more diverse notes for use in machine learning models.
Methods
The researchers implemented three different HIPS strategies for replacing PII: a standard Consistent replacement strategy which replaces all occurrences of PII with consistent surrogates; a Random replacement strategy which randomly assigns new values each time PII occurs; and a novel Markov chain based approach which uses natural language processing techniques such as part-of-speech tagging combined with statistical modeling techniques like ngrams in order generate realistic but randomized replacements that preserve syntactic structure within sentences while also ensuring no overlap between original values and generated surrogates.
Results
Using FNER ranging from 0.1% - 5%, the researchers observed that the Markov chain strategy significantly reduced PHI leakage compared to both the Consistent and Random strategies across multiple datasets including University Alabama Birmingham (UAB) discharge summaries corpus as well as MIMIC corpus consisting primarily discharge summaries from Beth Israel Deaconess Medical Center ICU patients over 10 years period . Specifically at 0.1% FNER document level leakage could be reduced from 27.1% down 0 1%, while at 5% FNER it could be reduced from 94 2% down 57 7%.
Discussion
The findings demonstrate that Markov chain based surrogate generation can substantially reduce risk associated with inadvertent release of confidential information across different assumed levels FNERs when compared against both consistent HIPS as well random HIPS strategies Furthermore authors released their implementation called BRATsynthetic on Github so other members clinical informatics community can benefit from their work
Conclusion The Markov chain substitution strategy allows larger deidentified corpora be released same risk level compared using consistent HIPS This provides potential increase availability clinically relevant data sets without compromising patient confidentiality
