Leveraging Cognitive and Speech Ecological Momentary Assessment in Individuals With Phenylketonuria: Development and Usability Study of Cognitive Fluctuations in a Rare Disease Population

Introduction

Phenylketonuria and Cognition

Each year, 6 out of 100,000 newborns are diagnosed with phenylketonuria (PKU) [1]. PKU is a rare, hereditary disease [2,3] that, if left untreated, can lead to severe brain damage, intellectual disabilities, and behavioral issues [1,3,4]. PKU is characterized by a deficiency in the phenylalanine hydroxylase enzyme, necessary for the metabolism of the amino acid or phenylalanine (Phe) [3,5-7]. This causes disruptions in Phe metabolism [2] and deficiencies in tyrosine, or Tyr, with significant downstream effects on serotonin and dopamine [5,7]. Individuals with PKU tend to experience difficulties on tests measuring verbal fluency [2,8], processing speed [3,9], and executive functioning [5,6]. In addition, despite early intervention, patients with PKU also typically exhibit lower IQ scores, but to a lesser degree than executive functioning and processing speed [1,3,4,9].

PKU Biomarkers and Cognitive Functioning

Cognitive functioning in patients with PKU is greatly affected by fluctuations in blood Phe and blood Tyr levels. Although the relationship between Phe and cognitive functioning is more like a continuum, such that higher Phe is associated with greater impairment in cognitive functioning, in general, patients with PKU with Phe levels above 600 mmol/L demonstrate impaired cognition [3,4] and poor frontal lobe function [10]. Participants with even higher Phe levels, above 1000 mmol/L, scored lower on a greater number of cognitive tests assessing cognitive domains; this includes attention, verbal fluency, reaction time, verbal recognition memory, visual memory, and naming, compared with those with Phe levels lower than 1000 mmol/L [11,12]. Greater variability in Phe levels also appears to contribute to the severity of neurocognitive sequelae [3]. Indeed, previous work points to variability in Phe control as the strongest predictor of executive function and general cognitive outcomes, wherein Phe variability may be a better indicator of cognitive functioning than both metabolic control and age [3].

Assessing Fluctuations in Cognitive Status

Most previous studies of neurocognitive functioning in individuals with PKU have relied on traditional neuropsychological evaluations, which only capture a single time point of assessment, or a “snapshot” of cognitive functioning. This is not necessarily problematic, as this is consistent with standard clinical practice; however, these assessments are often limited in that individuals with PKU may experience fluctuations in cognitive status, wherein their cognition may shift within hours, days, weeks, and months. Thus, a typical evaluation is unlikely to capture this variance adequately [13]. Thus, we use a methodological approach known as ecological momentary assessment (EMA) in this study; this facilitates repeated assessments over time to capture intraindividual and interindividual variability [14-16]. EMA has been particularly effective with the widespread use of smartphones, allowing real-time capture of behavioral, psychological, and cognitive processes. Furthermore, it is a particularly [17] robust method of data capture given that it enables investigators to account for contextual and environmental factors, which are typical limitations of traditional research methodologies that most frequently capture functioning in a lab at a single time point, without any external distractions [18,19].

Previous work has demonstrated that EMA can be used to evaluate everyday fluctuations in cognitive status, enabling assessment of cognitive functioning in naturalistic contexts [20,21]. Yet this approach is just recently being incorporated into studies with patients diagnosed with PKU, despite EMA being an effective tool to measure everyday variability, or fluctuations, in cognitive status [3,17,22]. EMA is an ideal methodology for the study of rare disease populations like PKU, because it allows for the collection of multiple data points from each individual by increasing reliability and improves statistical power to detect clinically meaningful findings with small sample sizes—a key factor in rare disease populations. In addition, EMA study designs enable detailed analyses of multiple dynamic biological, cognitive, and psychological processes, such that one can effectively characterize specific clinical populations [23-25].

This Study

The primary overarching goal of this study is to characterize the relationships among cognition, speech, mood, and blood-based biomarkers (Phe, Tyr) in individuals with early treated PKU. This study, led by principal investigator SS of McLean Hospital of Harvard Medical School and funded by the Phenylalanine Families and Researchers Exploring Evidence (PHEFREE) Consortium, leverages EMA to evaluate real-time cognitive status, speech or voice biomarkers, and psychological functioning. Blood-based biomarkers, including Phe and Tyr, are also useful in contextualizing these relationships and were assessed on the days of EMA administration; further work on this project will aim to gather more Phe and Tyr data to better establish psychometric robustness. Participants are continuing to be recruited for this study.

This paper reports results from an initial EMA optimization pilot of 20 participants diagnosed with PKU. This study was conducted before the completion of a longer protocol to determine the most optimal battery for individuals with PKU, including the number, length, and frequency of EMAs. We describe our initial optimization pilot results that are guiding the study design. We collected EMAs 6 times within a month; finger prick tests were completed on the day of each EMA to determine how variations in amino acid metabolism might relate to or predict fluctuations in certain aspects of functioning. This frequency was chosen to limit participant burden while still accounting for a wide range of data capture, varying between time of day and day of week over 1 month. Our goal is to determine whether the current method of evaluation, using EMA, is appropriate for individuals diagnosed with PKU, as well as how performance on cognitive tests might vary over time. This has implications for widespread dissemination of rapid, repeatable, scalable batteries that can be administered completely remotely, offering greater equity and accessibility in evaluating individuals with PKU on an international scale.

Methods

Participants

All participants were recruited through flyers distributed by the National PKU Alliance (NPKUA) through email with individuals in their patient registry and by postings on social media platforms associated with the organization. Recruitment specifically targeted individuals already enrolled in the NPKUA registry, a database that connects individuals diagnosed with PKU because it aligns with the study’s objective of examining the cognitive and behavioral impacts of the rare disease. Recruitment materials provided detailed information about the study and instructions for those who expressed interest in participation. To be eligible, participants had to meet the following inclusion criteria: current US resident, normal or corrected-to-normal vision, ability to provide consent, and a diagnosis of PKU. Participants were excluded based on the following: significant physical disabilities affecting their ability to perform digital assessments (eg, due to visual, motor, or hearing impairments) or their inability to complete EMAs during the study period (eg, due to planned travel, night shift work, or occupation that does not allow time to complete assessments within 60 minutes). Potential participants were screened during an initial virtual visit to confirm eligibility. The study’s purpose, procedures, risks, and expectations were thoroughly explained. Informed consent was collected electronically via a secure REDCap (Research Electronic Data Capture; Vanderbilt University) form before enrollment.

Materials

Baseline Assessment

Baseline cognitive tests, speech assessments, and psychological questionnaires were completed by all participants via their smartphones upon enrollment in the study. We used two platforms, including (1) TestMyBrain (TMB), which is an open-source, digital cognitive test platform with data collected from approximately 3 million participants worldwide [20,22,26-28], and (2) SurveyLex, developed by Sonde Health, which is a speech acquisition platform that enables voice recordings in response to predefined prompts or free response questions. This tool has been used to effectively validate voice biomarkers in other work [29]. Tasks were selected based on traditional neuropsychological evaluations, as well as speech measures typically used by collaborators on this project from the IBM Thomas J. Watson Research Center. In total, the baseline battery took approximately 90 minutes to complete and included “full versions” of all cognitive EMAs (refer to Table 1 for a complete list); of note, this is significantly longer than the ultrabrief, EMA versions of cognitive tests used for repeated assessments throughout the study. Please refer to Table 1 for a complete list of baseline assessments and their descriptions.

Cognitive EMA, Speech EMA, and Blood-Based Biomarkers

Overview

Ultrabrief versions of the TMB full length cognitive tests were selected for the optimization pilot; these versions of the full-length tests demonstrate robust reliability and good sensitivity [37]. These specific cognitive tests were chosen based on their domain of assessment; that is, given processing speed and executive functioning are implicated in PKU, brief versions of cognitive tests measuring these domains were chosen for the EMA study portion. All tests were developed using a combination of JavaScript and HTML, delivered through web apps that were downloaded to the participants local device, ran in the browser, and then delivered data back to a central server. Analyses required participants to complete at least 4 assessments, 1 of which was the baseline assessment. A brief description of the tests used are below, with further detailed information and psychometric characteristics described in Germine et al [38] and Singh et al [28].

TMB Digit Symbol Matching

In digit symbol matching (DSM) [22,26,39], participants are presented with 6 symbols, each of which are paired with a single digit between 1 and 3 (ie, 2 symbols were paired with each digit). These pairings remain visible throughout the duration of the test. Individual probe symbols are sequentially presented above these pairings, to which patients respond by selecting the corresponding digit as quickly as possible. Each probe symbol remains visible until the patient makes a response. Scores are recorded as the total number of correct responses in 90 seconds. The primary test score of interest used for psychometric analyses is number of correctly completed matches (DSM.score).

TMB Gradual Onset Continuous Performance Test

In gradual onset continuous performance test (gradCPT) [40], the participant presses a key when a city image appears and does not press it when a mountain image appears. Images rapidly transition from one to the next, with mountains appearing only 10%-20% of the time. Scores are recorded as a measure of response bias where a larger value indicates greater response impulsivity, or tendency to press a key regardless of the picture type. The primary test score of interest and used for psychometric analyses is response bias (CPT.dprime).

TMB Multiple Object Tracking Test

In multiple object tracking (MOT) test [41], the participant remembers and tracks a set of target circles as they move around the screen, among a larger set of identical distractor circles. The primary test score of interest and used for psychometric analyses is percent correct (MOT.score).

Speech EMA includes the same speech tests used in the baseline voice survey (refer to Table 1 for a comprehensive list, tests’ relative duration, and response style, ie, repetition or free response).

To determine Phe and Tyr levels, participants were supplied with test kits from PerkinElmer that were mailed to their homes. Participants provided blood samples that were mailed directly to PerkinElmer after obtaining a fasting sample via finger prick on the morning of their scheduled EMA.

In addition to cognitive EMA, speech EMA, and blood biomarkers, we also collected passive measures, including metadata about browser, screen size, and operating system. This information was used in data analysis to ensure consistent data quality throughout participant responses.

Procedure

This study was compliant with ethical principles and approved by the Mass General Brigham (MGB) Institutional Review Board (IRB) and the NPKUA Ethics Committee. All participants completed an orientation and signed the informed consent form through the secure REDCap platform. In total, 23 participants were recruited through flyers distributed by the NPKUA. Participants were asked to complete 6 EMAs over 1 month, varying by day of the week (weekday vs weekend) and time of day (morning, afternoon, or evening). Ultrabrief versions of selected baseline tasks, based on areas of functioning typically impaired in individuals with PKU, were administered on a mobile device at varying times throughout the day to minimize participant burden. More specifically, EMAs were sent on predetermined days throughout the month (week 1: Wednesday; week 2: Tuesday and Friday; week 3: Monday and Sunday; and week 4: Thursday). All participants followed the same EMA schedule in Eastern Standard Time: EMA 1 at 10:13 AM, EMA 2 at 10:13 AM, EMA 3 at 7:45 PM, EMA 4 at 12:05 PM, EMA 5 at 1:46 PM, and EMA 6 at 5:57 PM. There was a minimum of 3 days and a maximum of 6 days between EMAs. All EMAs were delivered between 9 AM and 9 PM local time to reduce interference with participants’ daily schedules and sleep routines. All participants completed EMAs on their personal devices (smartphones). They were routinely sent push notifications that enabled them to access the assessments via a web link; they had 60 minutes to complete the assessment. Participants were compensated US $300 (per EMA and blood collection). The between-person reliability of each EMA task was evaluated, which is especially relevant given that each task was administered in real-life settings with a clinical population. Refer to Figure 1 for the overall study design.

‎

The overall study design is the same as the optimization pilot discussed in this paper. Conducted over 1 month, the study includes a virtual prestudy visit conducted over Zoom (screening for inclusion and exclusion criteria, orientation, informed consent, and baseline assessment). From Days 1-6, participants complete repeated EMAs across three domains: (1) self-report EMAs that track variables such as argument occurrence, stress, COVID-19 impact, anxiety, depression, alertness, concentration, substance use, and sleep, (2) cognitive EMAs that include tasks like MOT, gradCPT, and DSM to measure cognitive function, and (3) speech EMAs include tasks such as short sentence repetition, sustained phonation, diadochokinetic exercises, paragraph reading, responses to “How do you feel?” prompts, and semantic fluency tasks.

Ethical Considerations

This study adhered to ethical guidelines for human participants research as outlined by McLean Hospital, Harvard Medical School, and MGB. Before study initiation, a formal review of the research protocol was conducted by the MGB IRB. Based on the nature of the study, the IRB determined that no exemptions were applicable. Once the study documents were approved (IRB approval number 2024P001922), all research activities were conducted in full compliance with institutional policies and federal regulations to uphold ethical standards.

Before providing consent, participants received a detailed informed consent form that outlined key components of the study, such as the purpose, procedures, risks, benefits, confidentiality, and privacy rights. Participants were given adequate time to review the consent form, ask questions, and address concerns with study staff before providing consent. The consent process was conducted virtually through the secure online platform REDCap.

To safeguard participant privacy and confidentiality, each participant was assigned a unique participant ID (eg, PKU1) generated in REDCap during the virtual consent process. Identifiable information was securely stored in password-protected files, accessible only to authorized study staff, and all data were deidentified. All TMB data were automatically backed up nightly, with access restricted to authorized users. Data collected via SurveyLex, a Health Insurance Portability and Accountability Act (HIPAA)–compliant software administered by Sonde Health, has been approved for use in other MGB IRB protocols (eg, 2019P003458 and 2019P002752). Any data that our team retrieved from the SurveyLex website were encrypted in transit and stored securely in a designated Partners Dropbox folder.

Participants were compensated up to US $300 for their participation in the study. Compensation included US $10 for each brief daily assessment (6 assessments × US $10 = US $60) and US $40 for each blood collection (6 collections × US $40 = US $240). Payments were issued via checks mailed to the participants’ homes.

Statistical Analyses

EMA and baseline tasks were parsed using Python 3.11 via Spyder IDE 5.4.3. Analysis was done using R 4.3.1 [42] in RStudio 2023.06.1+524 [43], with the packages tidyverse [44], plyr [45], and psych [46]. Plots were made with ggplot, part of tidyverse, as well as patchwork [47]. The between-person reliability of each EMA task was evaluated, which is especially relevant given that each task was administered in real-life settings with a clinical population. The between-person reliability, which represents the variance that is due to differences between individuals versus within individuals, was assessed using 2 different approaches: the mlr (multilevel reliability) function from the psych package in R (for gradCPT, MOT, and DSM) and a regression-based approach (for semantic fluency, due to the missing trial structure), modeled after Mascarenhas Fonseca et al [37]. The mlr function computes reliability as well as generalizability, using unconditional multilevel mixed models to predict performance for each EMA, with nested within-person random effects (between-person reliability: mlr ‘RkRn’; within-person reliability mlr ‘Rcn’ [24]). For gradCPT EMAs, response bias (dprime) was calculated for odd and even trials, and performance was predicted for the other half of the EMA. For DSM, scores were also calculated based on odd and even trials [27]. For MOT, scores were calculated on a trial-by-trial level [21] based on previous works [21,48]. EMA number and trial number (in case of MOT) or for both halves of the task (gradCPT and DSM) were coded from –2.5 to 2.5.

For the regression-based approach, using the lmer function in r, the calculated test scores were used and entered into an unconditional multilevel mixed model (equation 1) to predict scores on each EMA. Fitting the model allowed partitioning of variance, which could then be entered into equation 2 [37]:

semanticFluency<–lmer(score~1 + (1 + EMA_num|participant),data = df) (1)

EMA_num was a vector coded from –2.5 to 2.5 for EMAs 1 to 6.

Where Var (BP) is the total variance in scores between participants, Var (WP) is the variance in scores within participants (ie, variance between EMA sessions and residual variance), and n is the total number of measurements (in this case, the mean number of EMAs completed across all participants) [48]. Between-person reliability was reported for each EMA task, together with average completion rate and average EMA performance.

To assess variability in task performance (in percent) over the course of the EMAs, the coefficient of variability (CV) was calculated (equation 3) for each of the 3 tasks.

To make sure that we understand if performance was impacted by time of day, we used t tests to compute whether there was a significant difference in performance between morning EMAs (EMAs 1 and 2), evening (EMA 3 and 6), and morning and midday EMAs (EMA 4 and 5).

Finally, for baseline tasks, average performance was reported. Notably, although we collected self-reported mood and psychological data, given the small sample size, further data will need to be collected to sufficiently power additional analyses regarding the relationship among EMA and mood or self-report.

Results

Participants

In total, 21 adults with PKU were enrolled in this study between December 2022 and March 2023 (Table 2 illustrates descriptive statistics); however, only 20 participants were included in the EMA analyses due to 1 participant’s participation in a very similar study that was not disclosed before enrollment. Following the exclusion of that participant, 2 participants had missing baseline data, so 18 total participants were included in analyses. All 20 participants were diagnosed with PKU at or around the time of birth, and diagnosis details were self-reported. Among the participants, 19 had classic PKU (Phe >1200 mmol/L), 1 had hyperphenylalanemia (Phe 120-600 mmol/L), and 3 were unsure of their specific diagnosis type.

Baseline Results

Participants performed several baseline tasks that will be used for further analysis in subsequent publications as the study continues. Descriptive results are reported in Table 3 and presented alongside normative data from the TMB database for each test, respectively. Data in the table are reported as mean (SD) across the entire participant sample (for this study data as well as the normative sample). This is true for all score and accuracy data but also for the mean and median reaction times data.

Biomarkers

The averages for Phe, Tyr, and the Phe:Tyr ratio were recorded daily for each scheduled EMA. These blood-based biomarkers were measured in micromoles per liter (mmol/L), as illustrated in Table 4.

EMA Results

Participants were prompted to complete 4 tasks as part of each of the 6 EMAs (“measurement time points”) in the study. On average, each task was completed between 4.78 times out of the 6 measurement time points. Between-person reliability (ie, the consistency of the differences in scores between individuals [27]) was slightly lower or comparable to previous EMA studies [27,40] specifically for the TMB MOT and TMB DSM, and comparable with or exceeding a representative sample from TestMyBrain [40,49]. However, while reliability was slightly lower for the TMB gradCPT (0.72) and semantic fluency (0.70), they still fall within a good range (reliability between 0.4 and 0.59 is considered fair, 0.60 and 0.74 good, and reliability above 0.75 excellent). Results are listed in Table 5.

Figure 2 provides an overview of participant performance in all 4 tasks that were administered as part of an EMA, over the course of the 6 EMAs. Boxplots depict the data distribution, with the median shown as a horizontal line, the lower and upper quartile values as the edges of the box, while the whiskers represent the minimum and maximum data values. The blue points represent outliers (defined as the standard to be above or below the upper or lower quartile value minus 1.5 times the IQR) and the red points show individual participant values.

‎

Overview of participants’ performance on individual cognitive tasks administered as part of each EMA conducted over 1 month. The x-axes represent EMA time points from day 1 to day 6, and the y-axes represent participant performance for each task. Panel A shows performance on the gradCPT (n=20), with dprime values shown on the y-axis. The gradCPT assesses sustained attention, and participants are required to distinguish between target and nontarget stimuli. Dprime can be interpreted as the discrimination sensitivity in the task, with higher values indicating a better ability to perform the task. Panel B shows performance on the semantic fluency task (n=20), where the y-axis represents participants’ ability to produce words within a specific category (eg, animals) during a set time frame. A higher score indicates better semantic fluency. Panel C shows performance on the MOT (n=20), with the y-axis reflecting participants’ accuracy. The MOT assesses speeded visual attention, and participants are asked to track a set of target circles. A higher number indicates better accuracy, meaning how well participants tracked objects. Finally, panel D shows performance on the Digit Symbol Matching (DSM; n=20), with the y-axis representing the number of correctly completed matches. The DSM measures processing speed, where participants match symbols to corresponding digits as quickly as possible. The EMAs were sent on predetermined days throughout the month (week 1, EMA 1: Wednesday; week 2, EMA 2 and 3: Tuesday and Friday; week 3, EMA 4 and 5: Monday and Sunday; and week 4, EMA 6: Thursday). All participants followed the same EMA schedule in Eastern Standard Time: EMA 1 at 10:13 AM (gradCPT, MOT, and DSM, 17 out of 20 participants completed, verbal fluency: 16 out of 20 participants completed EMA), EMA 2 at 10:13 AM (gradCPT, MOT, and DSM, 18 out of 20 participants completed, verbal fluency: 16 out of 20 participants completed EMA), EMA 3 at 7:45 PM (gradCPT, MOT, and DSM, 18 out of 20 participants completed, verbal fluency: 14 out of 20 participants completed EMA), EMA 4 at 12:05 PM (gradCPT, MOT, and DSM, 16 out of 20 participants completed, verbal fluency: 15 out of 20 participants completed EMA), EMA 5 at 1:46 PM (gradCPT, MOT, and DSM, 16 out of 20 participants completed, verbal fluency: 16 out of 20 participants completed EMA), and EMA 6 at 5:57 PM (gradCPT, MOT, and DSM, 17 out of 20 participants completed, verbal fluency: 14 out of 20 participants completed EMA). There was a minimum of 3 days and a maximum of 6 days between EMAs. A total of 12 participants resided in the Eastern Time Zone, 4 in the Central Time Zone, 1 in the Mountain Time Zone, and 3 in the Pacific Time Zone.

Quantitative assessment of variability indicates a coefficient of variability of 28% for the gradCPT, 37% for semantic fluency, 15.8% for the MOT, and 17.6% for the DSM. Hence, performance is more stable in those tasks measuring processing speed, visual short-term memory, and visuospatial attention (DSM and MOT), while performance on the gradCPT (measuring sustained attention, response inhibition, and cognitive control) as well as semantic fluency (also measuring executive control) is much more variable over time. The semantic fluency prompt (ie, “name everything you can think of in this category”) was changed for each EMA, and therefore greater variability was expected for this speech-based task. This task also represents domains (eg, verbal fluency and aspects of executive functioning) known to be difficult for patients with PKU [50]. Performance in the TMB gradCPT is measured using dprime, which indicates the discrimination sensitivity in the task. Higher values reflect a better ability to perform the task. Performance in the TMB MOT is measured using accuracy, which assesses the ability to correctly track objects. Median performance across all EMAs is at 70%, indicating participants’ general ability to do well in the task. Furthermore, TMB DSM performance is measured using the DSM score, which indicates the number of correctly identified matches in the task. Higher values indicate a higher processing speed and better visual short-term memory. Median performance appears stable across all EMA measurements, with little fluctuation.

Finally, we assessed whether performance was significantly different between the morning (EMA 1 and 2) and evening (EMA 3 and 6) and morning and midday (EMA 4 and 5) EMAs. Results showed that for the DSM, there was no significant difference in performance between the morning and evening (t_13.72=1.46, P=.17) or morning and midday EMAs (t_13.90=–0.98, P=.34; average performance morning: mean 23, SD 3.76; midday: mean 24.94, SD 4.11; evening: mean 25.56, SD 3.26). For semantic fluency, we again did not see a difference in mean performance between morning (mean 19.2, SD 4.73) and evening (mean 19.3, SD 4.75) EMAs (t₈=–0.033, P=.97) and morning and midday (mean 20.4, SD 5.18) EMAs (t_7.93=–0.38, P=.71). A similar pattern emerges for the MOT, with no significant difference between morning (percent correct: mean 71.46, SD 19.35) and evening (mean 76.04, SD 19.54) EMAs (t₉₄=–1.16, P=.25) or morning and midday (mean 74.79, SD 19.46) EMAs (t₉₄=–0.84, P=.40). Finally, the same is true for the comparison between morning (dprime: mean 2.83, SD 0.32) and evening (mean 3.07, SD 0.51) EMAs in the gradCPT (t_11.68=–1.13, P=.29), and between the morning and midday (mean 3.17, SD 0.40) performance (t_13.25=–1.86, P=.09). Hence, performance does not decrease when EMAs are administered later in the day compared with earlier (all reported t tests are 2-tailed).

Attrition

In total, 21 participants were recruited through the NPKUA. Furthermore, 1 participant was removed because of a conflict of interest (they were a part of another study with a laboratory we are collaborating with at the University of Missouri-Columbia). All participants completed the study in its entirety. Regarding EMAs, completion rates were around 70% (between 4.6 and 5 measurements out of 6); 1 participant only completed 1 EMA and 2 blood samples.

Discussion

Principal Findings

Individuals with PKU face a large health care burden given that they endure multiple hospital visits from childhood, and must consistently monitor diet and blood levels, without the advent of continuous monitoring digital tools [51,52]. These individuals are more likely to experience chronic conditions of organ systems, further increasing their health care burden [52,53]. In addition, given that PKU is inherently a rare disease, there is often a dearth of specialty providers who can routinely monitor patients with the disease; thus, the time burden on caregivers and patients alike is further exacerbated [51]. Studies on this population need to focus on scalable, accessible ways of remotely monitoring individuals with PKU, so they can achieve a better quality of life with less health care burden. Thus, EMA studies in this population fill a unique gap, wherein providers and researchers alike can continuously monitor patients at scale, enabling greater access to individuals with PKU both nationally and internationally. Smartphones and the widespread availability of personal digital devices further facilitate larger-scale studies that can readily incorporate novel or experimental measures as they are developed.

The goal of this pilot study was to (1) demonstrate that EMA is a valid and reliable methodology for evaluating fluctuations in cognitive status in individuals diagnosed with PKU and (2) optimize a test battery that can be iterated on in the larger protocol based on these results. Given that this is the first study examining cognitive and speech EMA in individuals diagnosed with PKU, there were limited previous studies on which to develop appropriate frequency or timepoint decisions. Therefore, we turned to other clinical studies that use EMA in clinical and community samples [24,37]; based on the results demonstrating robust reliability (between-person for all tests and within-person for processing speed), it appears that this frequency can be deemed appropriate for this population.

Results suggest that EMAs were completed adequately well in this clinical sample, with completion rates above 70% (4.78 measurements out of 6). Furthermore, performance in both EMA measures and baseline tests appears in line with expectations based on expansive normative data, both for community and clinical samples, which demonstrate participants can complete these types of tasks with relatively low attrition or participant burden. Reliability is also close to what has been recorded in other samples, though slightly lower for the gradCPT and semantic fluency (0.7 and above), with the latter, however, representing a domain that is known to be specifically difficult for patients with PKU [53]. The TMB DSM and TMB MOT tests demonstrate the strongest between-person reliability in this sample, consistent with previous studies and the TMB representative sample [37].

Notably, there appeared to be small “dips” in semantic fluency on EMA 3; however, this qualitative decline cannot only be accounted for by time of assessment (ie, evening), given that EMA 6 was also in the evening, and time of day did not significantly impact performance. Furthermore, we did not see a significant difference between EMA 3 (mean 15.6, SD 3.85) and 6 (mean 23, SD 7.12; t_6.16=–2.05, P=.09). Similarly, TMB MOT appears to be somewhat more variable during EMA 3 as well, so we compared variability in performance on EMA 3 versus EMA 6, given that they are both administered in the evening, and there was no significant difference in performance (t_93.98=–0.28, P=.78) between EMA 3 (mean 75.41, SD 21.92) and EMA 6 (mean 76.67, SD 21.57). In addition, performance on the MOT during EMA 3 was in no way significantly different (Fligner-Killeen test of homogeneity of variances: P=.93). Therefore, it appears that there may be natural peaks and troughs in performance that, although not statistically significant, strengthen the argument for this methodology, which enables one to assess fluctuations in performance over one full month.

Limitations

Future work might examine whether higher-frequency EMAs could potentially offer greater utility in capturing nuanced changes in everyday cognition. Previous work by Mascarenhas Fonseca et al [37] has demonstrated that 3 EMAs per day for 10 days was particularly effective in capturing changes in glucose levels in individuals diagnosed with type 1 diabetes; this frequency should be examined within the PKU population, with finger prick tests completed at the time of EMA, rather than first thing in the morning, as done in this study, but no conclusions were drawn based on blood-based biomarkers given the limited range of Phe or Tyr. Ongoing analyses are determining whether any specific changes in blood levels within individuals may offer some interesting insights in relation to fluctuations in cognitive status. In addition, given the small sample size and relatively (racially or ethnically) homogeneous population, it is difficult to determine the generalizability of the results. All participants were recruited through the NPKUA, and therefore special efforts aimed at encouraging diversity were limited. Future work might attempt to recruit individuals with PKU from the broader community to better capture diversity. As this study continues, obtaining a larger sample size, though difficult in a rare disease population, will be critical in determining how cognition, other speech characteristics (besides semantic fluency), and blood-based biomarkers (Phe, Tyr) interact. Collecting this information will facilitate a data-driven approach in streamlining and refining the battery used in this study.

Conclusion

The EMA pilot study described in this manuscript demonstrated the psychometric reliability and feasibility of EMA studies in individuals with PKU. By leveraging digital tools, EMA offers the ability to remotely capture everyday cognitive functioning, outside of a single time point of assessment. The digital nature of EMA batteries facilitates entirely remote test administration, enabling more rapid and scalable patient monitoring while improving equity and accessibility, particularly in clinical trials interested in outcome-based research. Future projects may focus on validating a singular battery that could be rapidly administered at scale and in clinical trials to determine the progression of disease, with or without pharmacological intervention.