Zoom to the Virtual Room: The Shift to Remote Early Childhood Observational Assessments
Saved in:
| Title: | Zoom to the Virtual Room: The Shift to Remote Early Childhood Observational Assessments |
|---|---|
| Language: | English |
| Authors: | S. Darius Tandon (ORCID |
| Source: | Prevention Science. 2026 27(1):28-40. |
| Availability: | Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ |
| Peer Reviewed: | Y |
| Page Count: | 13 |
| Publication Date: | 2026 |
| Sponsoring Agency: | Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) (DHHS/NIH) |
| Contract Number: | RO1HD097215 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Early Childhood Education |
| Descriptors: | Early Childhood Education, Observation, Evaluation Methods, Distance Education, Videoconferencing, Virtual Classrooms, COVID-19, Pandemics, Children, Child Caregivers, Longitudinal Studies, Birth, Mothers, Depression (Psychology), Prevention, Intervention, Parenting Styles, Low Income Groups, Technology Uses in Education, Interrater Reliability, Home Visits, Preferences |
| DOI: | 10.1007/s11121-024-01737-z |
| ISSN: | 1389-4986 1573-6695 |
| Abstract: | The COVID-19 pandemic has led prevention researchers to increasingly use remote observational procedures given social distancing directives associated with the pandemic. However, few studies have used remote observational procedures with children and their caregivers, with scant literature describing procedures with socioeconomically, racially, and ethnically diverse families. This manuscript describes processes to pivot to remote assessment of parent and child observations in the context of a longitudinal study examining the effects of a postpartum depression preventive intervention on responsive parenting and child self-regulation. We conducted remote assessments across three timepoints--42, 48, and 54 months--with 133 low-income and racially and ethnically diverse parent--child dyads. Details are provided on remote observation preparation and setup, as well as adaptation of observational assessments. Lessons learned are shared on the use of technology, scheduling considerations, parent's role as facilitator, maintaining child engagement, and cost considerations. We demonstrated excellent inter-rater reliability between independent coders on all assessments, suggesting the quality of remote assessments was conducive for analysis. Surveys with families completing a remote assessment found that most felt it was easy to participate in remote assessments and their child had a positive experience. Most parents preferred virtual visits if given an option for future assessments. Much prevention research utilizes observational measures that are less subject to bias. Our study demonstrated that research teams can effectively and reliably pivot to remote assessment with racially and ethnically diverse, low socioeconomic families, thereby providing guidance to other prevention researchers considering similar remote assessments with diverse samples. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1508200 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHS5HuVYi7O5IBq7BHljyDtAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDGP43yctYhbqijs6HQIBEICBmgSeXNBhwGd9liHI3wIK5Xxs3eBiZNKtw2tTfhsPrptrFfz3BvMMvC38YTK97MTSDB1GEXyvId9Zjn9gTYa1MXkxvo-4tDI5dQYQDeQVN7mdB0wPBFRcY74IskBRjiBU8YxF3cd_hEbIfoZbdXKtfLpaorVTdcMQ87SrPeoWFaVYDhVCZSVF19WKCEfCcHWgwpF7WMjconXCqi4= Text: Availability: 1 Value: <anid>AN0191574884;n9p01jan.26;2026Feb17.02:13;v2.2.500</anid> <title id="AN0191574884-1">Zoom to the Virtual Room: The Shift to Remote Early Childhood Observational Assessments </title> <p>The COVID-19 pandemic has led prevention researchers to increasingly use remote observational procedures given social distancing directives associated with the pandemic. However, few studies have used remote observational procedures with children and their caregivers, with scant literature describing procedures with socioeconomically, racially, and ethnically diverse families. This manuscript describes processes to pivot to remote assessment of parent and child observations in the context of a longitudinal study examining the effects of a postpartum depression preventive intervention on responsive parenting and child self-regulation. We conducted remote assessments across three timepoints—42, 48, and 54 months—with 133 low-income and racially and ethnically diverse parent–child dyads. Details are provided on remote observation preparation and setup, as well as adaptation of observational assessments. Lessons learned are shared on the use of technology, scheduling considerations, parent's role as facilitator, maintaining child engagement, and cost considerations. We demonstrated excellent inter-rater reliability between independent coders on all assessments, suggesting the quality of remote assessments was conducive for analysis. Surveys with families completing a remote assessment found that most felt it was easy to participate in remote assessments and their child had a positive experience. Most parents preferred virtual visits if given an option for future assessments. Much prevention research utilizes observational measures that are less subject to bias. Our study demonstrated that research teams can effectively and reliably pivot to remote assessment with racially and ethnically diverse, low socioeconomic families, thereby providing guidance to other prevention researchers considering similar remote assessments with diverse samples.</p> <p>Keywords: COVID-19; Virtual assessment; Child self-regulation; Parenting; Medical and Health Sciences Public Health and Health Services Psychology and Cognitive Sciences Psychology</p> <p>Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s11121-024-01737-z.</p> <p>This manuscript adds to the growing literature describing remote observation research protocols developed in response to social distancing and stay-at-home directives associated with COVID-19. Several factors have been highlighted as potential barriers to successful administration of remote assessments in children and their caregivers, including the need to match tasks to children's language and comprehension, fatigue during assessments, technology limitations and variable access to technology, and the potential for interruptions when outside of a controlled laboratory environment (Shields et al., [<reflink idref="bib24" id="ref1">24</reflink>]; Zaadnoordijk &amp; Cusack, [<reflink idref="bib28" id="ref2">28</reflink>]). Given the societal shifts resulting from the COVID-19 pandemic that have led to increased comfort and reliance on remote interactions and adults' access to digital technology in the USA (Pew Research Center, [<reflink idref="bib19" id="ref3">19</reflink>]), modifications to research protocols have followed suit.</p> <p>There is a small, yet growing literature that describes modifications made by research teams to employ remote observational assessments with children and their caregivers (e.g., García-Álvarez et al., [<reflink idref="bib9" id="ref4">9</reflink>]; Gijbels et al., [<reflink idref="bib10" id="ref5">10</reflink>]; Krogh-Jespersen et al., [<reflink idref="bib13" id="ref6">13</reflink>]; Manning et al., [<reflink idref="bib14" id="ref7">14</reflink>]; McElwain et al., [<reflink idref="bib15" id="ref8">15</reflink>]; Schein et al., [<reflink idref="bib22" id="ref9">22</reflink>]; Shin et al., [<reflink idref="bib25" id="ref10">25</reflink>]; Segal &amp; Moulson, [<reflink idref="bib23" id="ref11">23</reflink>]), although literature illustrating efforts to conduct remote assessments with families who are socioeconomically, racially, and ethnically diverse are sparse. Thus, while previous has demonstrated feasibility in adapting research instruments and protocols for use with more affluent non-minoritized samples, a highly unique aspect of the present study is our pivot to remote observation with a low socioeconomic, racially and ethnically diverse, bilingual (English and Spanish) sample. The specific aims of this manuscript are to (<reflink idref="bib1" id="ref12">1</reflink>) describe the processes taken by our research lab to pivot to remote assessment of parent and child observations; (<reflink idref="bib2" id="ref13">2</reflink>) share lessons learned in developing and implementing our remote observations, and (<reflink idref="bib3" id="ref14">3</reflink>) present data demonstrating feasibility and acceptability of remote observations, indicated by quality of observational videos, reliability of assessment coding and participants' reflections on conducting remote assessments.</p> <hd id="AN0191574884-2">Study Background and Context</hd> <p>Recent reviews, including one conducted by the United States Preventive Services Taskforce (USPSTF), concluded that evidence-based interventions exist which prevent the onset and worsening of depression among pregnant individuals and new mothers (US Preventive Services Task Force et al., [<reflink idref="bib26" id="ref15">26</reflink>]; Pettman et al., [<reflink idref="bib18" id="ref16">18</reflink>]). One recommended intervention identified by the USPSTF was the Mothers and Babies Course (MB), a cognitive-behavioral intervention that has prevented the onset of new cases of major depressive disorder and led to reductions in depressive symptoms. Randomized controlled trials have yielded preliminary evidence of MB's impact on strengthening central facets of the caregiver-child relationship that promote adaptive outcomes and resilience including maternal responsiveness and secure maternal-child attachment (Alhusen et al., [<reflink idref="bib1" id="ref17">1</reflink>]; McFarlane et al., [<reflink idref="bib16" id="ref18">16</reflink>]).</p> <p>Conceptually, MB targets and has been demonstrated to improve core cognitive-behavioral therapy (CBT) skills including awareness of thoughts and decentering, behavioral activation, mood regulation, and use of social support, which are often deficient in mothers with depressive symptoms. These same skills are thought to underlie a mother's ability to engage in responsive parenting practices that promote young children's self-regulation. Specifically, these skills are thought to increase responsiveness to the child's needs and a mother's ability to create an affectively positive and cognitively stimulating environment, in which stressful events and hostile/reactive responses to the child are minimized. As such, we conducted a longitudinal observational study—Effects of a Prenatal Depression Preventive Intervention on Parenting and Young Child's Self-Regulation and Functioning (EPIC)—with the primary hypothesis that receipt of MB while pregnant would prevent maladaptive parenting and poor child self-regulation among intervention recipients when compared to mothers and children who did not receive MB (Diebold et al., [<reflink idref="bib7" id="ref19">7</reflink>]).</p> <p>EPIC built on a previously funded comparative effectiveness study that enrolled nearly 900 families from 37 community-based home visiting (HV) programs across seven midwestern states (Jensen et al., [<reflink idref="bib12" id="ref20">12</reflink>]). This comparative effectiveness study clusters randomized HV programs to receive MB delivered prenatally by mental health clinicians, MB delivered prenatally by lay home visitors, or usual HV services. By virtue of the large sample from the parent study, EPIC is poised to build on earlier MB effectiveness research on maternal responsiveness by examining multiple constructs of parenting and child self-regulation among a large, racially and ethnically diverse sample.</p> <p>EPIC was founded in late 2019 and aims to examine (<reflink idref="bib1" id="ref21">1</reflink>) the effects of MB on parenting and child self-regulatory skill acquisition through 54 months of age; (<reflink idref="bib2" id="ref22">2</reflink>) maternal mechanisms by which MB influences parenting and child self-regulation; and (<reflink idref="bib3" id="ref23">3</reflink>) whether fathers' contributions and key sociodemographic factors moderate MB impact on maternal parenting and child's self-regulation (Diebold et al., [<reflink idref="bib7" id="ref24">7</reflink>]). All EPIC study participants were asked to complete maternal report assessments, with a subset of participants also asked to participate in an observational sub-study that conducted objective measurements of parenting responsiveness and child self-regulation. In-person observational assessments were proposed to begin in 2020, which became impractical given the COVID-19 pandemic and social distancing mandates. Proposed observational assessments included both direct child developmental assessments and parent–child interaction. Given the importance of conducting objective measurements to assess our primary outcomes of parenting responsiveness and child self-regulation, we determined that a pivot to conducting virtual parent and child observation would be most conducive to achieving our study aims. This switch to virtual observation is consistent with recent trends to use remote methods given the growing recognition that remote research may allow for quicker data collection and inclusion of more diverse samples (Krogh-Jespersen et al., [<reflink idref="bib13" id="ref25">13</reflink>]). The present study focuses only on the process and feasibility of obtaining parent–child outcomes remotely. Future publications will report on the impact of the EPIC intervention on these outcomes.</p> <hd id="AN0191574884-3">Methods</hd> <p></p> <hd id="AN0191574884-4">Study Participants</hd> <p>EPIC eligibility criteria were ≥ 18 years old, English- or Spanish-speaking, and being the parent of a child turning 30 months old during recruitment period. Enrolled participants were provided the opportunity to refer the child's father or other adult caregiver at enrollment and at each subsequent survey timepoint. In total, 960 participants were enrolled in EPIC (509 from the parent study, 126 from the newly partnering HV programs in Illinois, and 325 fathers/caregivers).</p> <p>For the observational sub-study that forms the basis of the work described in this manuscript, enrolled maternal participants living in Illinois at time of enrollment were asked to participate in videorecorded parent–child observational assessments when their children were 42, 48, and 54 months old. Observations were limited to Illinois for logistical and budgetary reasons, as observations were initially proposed to be conducted in-person. In March 2020, when COVID-19 stay-at-home guidelines were first implemented, no observations had been completed. The study team discussed options for completing parent and child observations in light of the stay-at-home guidance and, faced with uncertainty regarding the timeline for a return to in-person contact, the decision was made to modify study procedures from in-person to virtual observation.</p> <p>Of the 205 participants living in Illinois contacted to complete a virtual observational assessment, 133 (65%) agreed and completed at least one remote assessment. When comparing demographics of those completing at least one assessment with those not completing any assessments, individuals who were married and not receiving any type of mental health services were more likely to complete an assessment than individuals who were not married or receiving mental health services, respectively. Of the 133 participants, 73 were in the MB intervention group during the parent study and 60 were control participants. Most participants in the observational sub-study identified as Black/African American (38%) or White (31%); over half (56%) reported a household income of less than $25,000; the mean age was 30 years old; and the majority (82%) had more than one child (see Table 1 for full demographic information). Participants were geographically dispersed throughout Illinois. To obtain participant perspectives on virtual observations, 37 participants who completed their final assessment between October 2022 and March 2023 were asked to complete an Observational Feedback Survey to describe their experience with the virtual observational assessment. Of these 37 participants, 33 returned their surveys, including 22 English-speaking and 11 Spanish-speaking participants.</p> <p>Table 1 Participant demographics of the observation sample (<emph>N</emph> = 133)</p> <p> <ephtml> &lt;table rules="groups"&gt;&lt;thead&gt;&lt;tr&gt;&lt;th align="left"&gt;&lt;p&gt;Demographic characteristic&lt;/p&gt;&lt;/th&gt;&lt;th align="left"&gt;&lt;p&gt;Total (&lt;italic&gt;n&lt;/italic&gt;)&lt;/p&gt;&lt;/th&gt;&lt;th align="left"&gt;&lt;p&gt;Percentage (%)&lt;/p&gt;&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Married or engaged&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;64&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;48.12&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Education level&lt;/p&gt;&lt;/td&gt;&lt;td align="left" /&gt;&lt;td char="." align="char" /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; &amp;#60; 12th grade&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;37&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;27.82&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; High school diploma/GED&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;31&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;23.31&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; Vocational school/training&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;12&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;9.02&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; Some college, no degree&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;35&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;26.32&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; College degree or beyond&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;18&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;13.53&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;More than one child&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;109&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;81.95&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Part- or full-time employment&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;57&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;42.85&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Race&lt;/p&gt;&lt;/td&gt;&lt;td align="left" /&gt;&lt;td char="." align="char" /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; Black/African American&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;51&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;38.35&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; White&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;41&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;30.83&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; Other (Asian, AI/AN, Native Hawaiian/Other Pacific Islander, Bi/multi-racial)&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;8&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;6.01&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; Other (not specified)&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;30&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;22.56&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt; Missing&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;3&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;2.26&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Hispanic ethnicity&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;55&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;41.35&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Household income &amp;#60; $25,000&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;74&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;55.64&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Age (mean/SD)&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;30.37&lt;/p&gt;&lt;/td&gt;&lt;td char="." align="char"&gt;&lt;p&gt;(5.97)&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <hd id="AN0191574884-5">Procedures</hd> <p></p> <hd id="AN0191574884-6">Observational Assessment Procedures</hd> <p>A total of 311 remote observations were completed across three timepoints (<reflink idref="bib42" id="ref26">42</reflink>, 48, 54 months) among the 133 parent–child dyads. Of the 311 total remote observations completed, 212 were with English-speaking participants and 99 were with Spanish-speaking participants. Each observation time point, which consisted of multiple assessments, was scheduled for 2 h. A total of 117, 104, and 90 visits were completed at the first, second, and third timepoints, respectively. Attempts were made to conduct observations at each timepoint, irrespective of whether the previous observation had been completed. Participants received $35–$45 gift cards for each observation completed, with amounts increasing for later timepoints. This remuneration included $10 provided at each timepoint to help offset data usage. Observations were conducted by four trained assessors, including two bilingual Spanish speakers. The assessors were members of the research team and had prior experience working with families and young children.</p> <p>Selected participants from the EPIC sample were sent a recruitment flyer via text or email, depending on their contact preference. Before the first timepoint, assessors completed a scheduling call for interested participants and screened for eligibility, which included seeing their child at least once a month. All subsequent visits were scheduled through text message or email based on contact preference. Scheduling protocols were designed to be as flexible as possible during the eligibility window, including evening and weekend options.</p> <hd id="AN0191574884-7">Remote Observation Preparation and Setup</hd> <p>The shift to remote observational assessments required significant adjustments to the original study protocol. Having access to a device with a microphone, camera, and the internet was added to the eligibility criteria for participating in the observational sub-study. Additional training was given to assessors to prepare them for remote administration. In particular, assessors were trained on Zoom functionality and navigation of equipment used during the remote observation (e.g., sharing multiple documents for each assessment, managing multiple devices, completing scoresheets). These technical competencies ensured that assessors adhered to each assessment's standardized protocols. With remote work, assessors required more technological tools. In addition to laptops, all assessors were provided an additional monitor, headphones, and wireless hotspot devices to strengthen their Internet connections when working from home.</p> <p>In the remote setup, all required materials for assessments were packed in a visit box and shipped directly to participants' homes or other preferred locations—a cost incurred due to virtual administration. Due to the number and geographical spread of participants, and concerns with disinfecting materials during the COVID-19 pandemic, it was determined that a separate visit box would be sent to each family, rather than attempting to reuse materials between families. To ensure novelty of the materials, participants were explicitly asked to refrain from opening the visit box until the visit but were told they would be able to keep all materials. A colorful, printed reminder was included at the top of the visit box, to remind participants not to open the box early. For the most part, caregivers were successful in complying with this request. Assessors monitored tracking information and confirmed delivery with all participants. Assessors also troubleshooted shipping issues while maintaining close communication with participants regarding delivery updates.</p> <p>To accommodate the technological needs of each participant, assessors provided device-specific guidance. After scheduling, participants received a comprehensive step-by-step instructional digital pamphlet for downloading the Zoom app and installing it on their devices. It was essential that participants download the Zoom app to participate, as the web-based platform did not display the stimuli pages for cognitive testing adequately, and assessors routinely checked to ensure this had been done at the time of the visit. Participants were also given the opportunity to test a Zoom call with an assessor if they were unfamiliar with their device settings. For privacy purposes, assessors did not provide Zoom invitations for the actual assessment to participants until the day of the visit and did not begin Zoom recordings until participants indicated readiness. Participants joined the Zoom calls on laptops, Chromebooks, smartphones, and tablets, and were allowed to switch devices between assessments to increase ease and comfort. For instance, participants changed devices for battery-related purposes, or switched to a new device that supported Zoom annotations for specific assessments. The Zoom annotation feature allows children to "draw" while content is shared on the screen. The markings are made directly by touching the touchpad or touchscreen. All participants were advised to have a charging source nearby in case it was needed during the Zoom call.</p> <p>At the start of each visit, assessors spent 10 min with each parent to make necessary Zoom setting adjustments and review visit guidelines. During this time, assessors confirmed participants' preferred contact information in case participants were disconnected from the call. Additionally, assessors inquired about Zoom connection quality, addressing any potential concerns related to Zoom functionality or Wi-Fi connectivity. If using a smartphone or tablet, participants were asked to place their devices in landscape view to maximize the visibility of assessment content. Assessors administered the Device Use and Familiarity Survey during this initial check-in, which included questions about what device would be used for the observation, how often the child had used that device in the last 3 months, and how often the child had participated in video calls in the last 3 months. The responses to these questions were used by assessors to understand how much support would be required during the visit, recognizing that some children are familiar with device use and video calls. During this setup period, assessors reviewed a "parent cheat sheet" that was sent in the visit box. The cheat sheet was a visually appealing and simple list of "dos" and "don'ts" designed to help parents understand what was appropriate to say to facilitate their child's engagement during the assessments (Supplemental Material). Assessors advised parents to refer to this sheet throughout the visit when unsure how to encourage their children's participation. The cheat sheet was helpful as it provided verbatim guidance for parents. Specific guidance regarding providing feedback to their child was given to parents before each assessment.</p> <p>For each assessment administered during the virtual observations, assessors guided parents through Zoom setting adjustments and, if necessary, shared PowerPoints that provided additional visual guidance. The step-by-step PowerPoints were helpful for participants who did not understand what to adjust (e.g., hiding "self-view," minimizing the thumbnail, moving the Zoom control bar so that the shared stimuli were not blocked, and Zoom annotations). PowerPoints were specific to device type and preferred language. Providing step-by-step instructions and visual guidance throughout the visit was fundamental for efficient assessment administrations and ensured the support of participants with a range of technology familiarity.</p> <hd id="AN0191574884-8">Observational Feedback Survey Procedures</hd> <p>Developed by investigators, this survey included 23 questions in the following domains: ease of participation, technology, demands of the visits, child engagement and performance, and preference for future observations (see Supplemental Material for copy of survey). All items were measured on a 5-degree scale of agreement (1 = strongly disagree, 5 = strongly agree). Feedback surveys were immediately sent to participants after their visit was completed and they were reminded to complete the survey 1 week later. Participants who completed the survey within 2 weeks were entered into a raffle to win one of two $100 gift cards.</p> <hd id="AN0191574884-9">Instrument Descriptions</hd> <p>All measures were designed for in-person administration with setup, instructions, and administration protocols virtually adapted for each assessment. Below, we describe each measure and adaptations made to facilitate their use via remote administration. Investigators deliberated the order assessments were conducted and chose to administer the Affect Test of Knowledge (AKT), Minnesota Executive Functioning Scale (MEFS), and Bracken School Readiness Assessment-Third Edition (BSRA-3), followed by the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS) and Brief Early Relational Assessment (B-ERA). We started with the AKT because it was most engaging to children. The MEFS was second because it was most challenging for children and parents due to the need for sustained focus. The BSRA-3 was more forgiving in terms of need for sustained focus and, therefore, placed after the MEFS to give the dyad a less intensive assessment after the MEFS (see Table 2 for an overview of the adaptations made to accommodate remote administration).</p> <p>Table 2 Remote administration adaptations, challenges, and lessons learned, by instrument</p> <p> <ephtml> &lt;table rules="groups"&gt;&lt;thead&gt;&lt;tr&gt;&lt;th align="left"&gt;&lt;p&gt;Instrument&lt;/p&gt;&lt;/th&gt;&lt;th align="left"&gt;&lt;p&gt;Adaptations for remote administration&lt;/p&gt;&lt;/th&gt;&lt;th align="left"&gt;&lt;p&gt;Challenges and lessons learned of remote administration&lt;/p&gt;&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Social-emotional measure&lt;/p&gt;&lt;/td&gt;&lt;td align="left" /&gt;&lt;td align="left" /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;AKT&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Created PowerPoint presentations displaying the four faces used in the Expressive and Receptive tasks&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Starting remote assessments with expressive task&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left" /&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Pre-recorded skits were integrated into PowerPoint for the Puppet task&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; PowerPoints loaded slowly due to video skit file size; assessors opened ahead of visit&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left" /&gt;&lt;td align="left" /&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Retaining children's attention to watch the videos and re-playing if necessary&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Executive function measure&lt;/p&gt;&lt;/td&gt;&lt;td align="left" /&gt;&lt;td align="left" /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;MEFS&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; To share MEFS app content, the tablet "joined" Zoom calls and shared screen&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Increased administration time&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left" /&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Unable to sort the cards by dragging them on the tablet, children pointed and parent relayed their response&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Child's frustration with not being able to move cards on one's own&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left" /&gt;&lt;td align="left" /&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Frequent reminders are needed for parents to correctly relay children's responses without providing feedback/guidance&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;School readiness measure&lt;/p&gt;&lt;/td&gt;&lt;td align="left" /&gt;&lt;td align="left" /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;BSRA-3&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Stimuli books were scanned into PDFs and shared on Zoom calls&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Children are distracted by color and style options on Zoom leading assessors to disable Zoom annotations&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left" /&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Children marked their responses utilizing Zoom annotations&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Frequent reminders are needed for parents to correctly relay children's responses without providing feedback when Zoom annotations unavailable&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;Parent&amp;#8211;child behavioral coding measures&lt;/p&gt;&lt;/td&gt;&lt;td align="left" /&gt;&lt;td align="left" /&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;DB-DOS&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Packaged and shipped activity materials to participants&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Parents helped prevent interference from siblings during assessment activities, and research team sent extra materials for siblings&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;P-COS&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Removed Puzzle task&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Assessors and parents determined ideal location for activities and device setup&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td align="left"&gt;&lt;p&gt;B-ERA&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Utilize free-play section of DB-DOS&lt;/p&gt;&lt;/td&gt;&lt;td align="left"&gt;&lt;p&gt;&amp;#8226; Unable to reuse materials&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <hd id="AN0191574884-10">Affect Test of Knowledge (AKT)</hd> <p>The AKT assesses preschoolers' understanding of emotions using felt fabric faces and puppets through four tasks (Denham et al., [<reflink idref="bib5" id="ref27">5</reflink>]). In the Expressive task, the child is asked to name each of the four faces (happy, sad, angry, afraid). In the Receptive task, the child is asked to point to the face with the emotion that the assessor prompts. In the Teaching task, the assessor demonstrates the faces' emotional state through labeling, facial expressions, and tone. In the Puppet Task, the assessor enacts skits in which the child is then asked to state how the main character feels. Children can verbally respond to identify the emotion or point to one of the four faces. The main character in the puppet skits mirrors the children's race and gender. In the first three puppet skits, all children observe the same situations that depict sad, angry, and afraid emotional states. The last six skits have two versions of the puppet's emotional response and the skit is determined based on parental indication of how their child would react in those situations. The assessor enacts the skit with the opposite parent's response to determine if children can identify emotions outside of their own perspectives.</p> <hd id="AN0191574884-11">Adaptation of the AKT</hd> <p>All modifications were made after consultations with AKT developers. While adapting the AKT for this study, a computerized version of the AKT was in development but not yet ready for release (Denham et al., [<reflink idref="bib6" id="ref28">6</reflink>]); the computerized version featured pre-recorded puppet videos, for example, with our adaptations building on many of the features of the computerized version. To ensure validity and reliability, all assessors were trained to assure they properly shared all materials to maximize clarity and sound quality. This was essential for the Puppet Task as Zoom controls often interfered while the videos played. In addition, parents were asked to relay children's responses using the numbers underneath each pictured face for standardization purposes. During the puppet skits, parents were also able to hold up the face that the child pointed to if it was unclear for assessors. AKT administrations were in Spanish or English, including the puppet pre-recorded skits.</p> <p>In-person administration includes randomly shuffling felt faces in front of the child while the assessor points to the face of interest and asks the child to label the emotion (expressive task) or point to the emotion the assessor labels (receptive task). In the virtual adaptation, four PowerPoint versions were made to include the four faces in different order for the Expressive and Receptive tasks. In the Expressive task, the parent was asked to point to the numbered face when the assessor asked the child to name the emotion it showed. This was to help ensure that the child was looking at the correct face. In the Receptive task, parents were asked to verbally relay the numbered face that the child chose after the assessor prompted the child to identify a specific emotion. All faces in the PowerPoint versions were numbered so that parents did not label any emotion or indicate correctness. The Teaching task was not different from the in-person administration but required proper lighting and sound from the assessor's technology to assure that the demonstration was seen and heard clearly. Live administration of this task was crucial to assure that children understood each emotion before completing the last Puppet task. All puppet skits were recorded and organized in PowerPoints to mirror children's race and gender with all skit versions. During the initial setup after welcoming the dyad to the call, assessors completed the skit survey with the parents to know which version to play as they moved through the PowerPoint.</p> <hd id="AN0191574884-12">Minnesota Executive Functioning Scale (MEFS)</hd> <p>The MEFS App is a standardized, table-based card-sorting game that measures executive functioning (EF), the neurocognitive skills that serve as the foundation for early learning such as attention, memory, control over impulsive thoughts and behaviors (Reflection Sciences, [<reflink idref="bib20" id="ref29">20</reflink>]). There are seven levels with increasing difficultness based on the cards' images and sorting rules. It is designed for ages two through adulthood as it is adaptive to each participant's current EF level. Scoring is automated and normed on a sample of 32,800 US children and adults. The MEFS app provides a script for all instructions and prompts.</p> <hd id="AN0191574884-13">Adaptation of the MEFS</hd> <p>A remote MEFS protocol had been posted on the MEFS developers (Reflection Sciences, Inc.) website, and all modifications to our protocol were made after consultations with the MEFS developers. All MEFS administrations were either in Spanish or English. During in-person administration, the assessor shares a tablet with the child while reading instructions and prompts the child by manually dragging the cards on the screen. In the remote administration, the tablet joined the Zoom call as a guest and shared screen so the child could see the MEFS app screen. After sharing the MEFS app screen on Zoom, assessors read prompts while parents verbally relayed where the child pointed, and the assessor manually sorted on the tablet. Assessors instruct parents how to label the sorting boxes for each level while refraining from using "correct," "right," or "left box," language.</p> <p>All scripted language on the MEFS app was designed for in-person administration, therefore, prompts were modified to instruct children to point instead of dragging the cards; prompts had been generated by the MEFS developers. To ensure validity and reliability, additional guidance was provided to parents before administration so that they did not repeat instructions or guide children's sorting. Parents were consistently reminded how to label boxes when verbally relaying responses so they did not indicate correctness or provide sorting feedback. In-person administration lasted 2 to 6 min but lasted up to 20 min during remote administration due to technical setup, additional instructions, and time spent entering relayed responses.</p> <hd id="AN0191574884-14">Bracken School Readiness Assessment-Third Edition (BSRA-3)</hd> <p>The BSRA-3 is a measure of school readiness for children ages 3 to 6 years old (Bracken &amp; Panter, [<reflink idref="bib2" id="ref30">2</reflink>]). It consists of five concept knowledge subsets, including colors, letters, numbers, sizes and comparisons, and shapes. For each subset, the child is asked to identify items until they reach a ceiling level. Their score is compared to a nationally normed sample for English speakers.</p> <hd id="AN0191574884-15">Adaptation of the BSRA-3</hd> <p>Pearson granted permission to scan the stimulus book to PDF for virtual visits since the BSRA-3 was not available as a digital tool at the time of this study. In Fall (2022), Pearson developed the Bracken Basic Concept Scale Fourth Edition Receptive (BBCS-4:R) that can be used online with automated scoring (Bracken, [<reflink idref="bib3" id="ref31">3</reflink>]). All BSRA-3 administrations were in Spanish or English. Specifically, the BSRA-3 was completed in the child's dominant language, but missed items were prompted in a second pass in the opposite language.</p> <p>For in-person administration, the assessor presents stimuli in a book while the child points. During remote administration, stimuli book pages were scanned into PDFs and shared during the Zoom call with permission from the publisher. Zoom annotations allowed the child to directly indicate their response on the screen. For instances when devices did not support Zoom annotation or parents were unable to enable annotations, parents verbally relayed children's responses. For participants that completed the visit on laptop and desktop computers, parents were asked to manage annotations with their mousepads after children pointed to their responses. Parents were provided additional guidance on Zoom annotations specific to their device and children trialed with practice items to become familiar with annotating. Additionally, assessors were trained to manage Zoom annotations so it was not distracting to the child. When Zoom annotations were unavailable, parents were asked to explicitly state their children's response as the item they chose. If the child was asked to show the number "15" but chose "5," parents had to verbally relay "5" and not redirect or prompt the child.</p> <hd id="AN0191574884-16">The Disruptive Behavior Diagnostic Observation Schedule (DB-DOS)</hd> <p>The DB-DOS is a well-validated observational tool to efficiently capture child behavior/regulation within varied interactional contexts–specifically with a parent/primary caregiver and with an examiner (Wakschlag et al., [<reflink idref="bib27" id="ref32">27</reflink>]). For this study, only the parent context was used. A key feature of the DB-DOS is its use of activity-based "presses" to rapidly elicit individual differences in children's capacity to regulate negative emotions in response to varied demands. It includes a series of semi-structured activities that the parent and child complete together, ranging from potentially fun to potentially frustrating activities including "do" and "don't" compliance tasks. It helps distinguish disruptive behaviors from normative misbehaviors of early childhood through behavioral coding on a systematic scoring method of clinical observations.</p> <hd id="AN0191574884-17">Adaptation to the DB-DOS</hd> <p>The DB-DOS was shortened for remote administration. For this study, the DB-DOS consisted of three activities lasting 15–20 min compared to 45 min for in-person administration. The DB-DOS includes activities involving interactions between an examiner and child as well as between caregiver and child. Due to limitations of meaningful interactions between examiners and children achieved virtually, remote DB-DOS administration only included caregiver-child activities. The shortened version retained tasks that "pressed" the child for frustration and irritability, but were more feasible for remote administration (Krogh-Jespersen et al., [<reflink idref="bib13" id="ref33">13</reflink>]). The same tasks used to assess child self-regulation are used for assessing maternal positive and negative emotions and discipline strategies, using a companion parenting coding system, the Parenting Clinical Observation Schedule (P-COS, Wakschlag et al., [<reflink idref="bib27" id="ref34">27</reflink>]).</p> <p>Virtual adaptation included careful packing of materials for participants to easily locate during each task, which required the most setup from parents. Prior to administering, assessors spent time guiding parents on how to place materials and devices so they could be seen and heard clearly throughout tasks. After activities began, assessors assured that the dyad was always visible on the screen and monitored lighting and sound quality, asking parents to make necessary accommodations. Unlike lab or at-home administrations, assessors had limited control over the device used or the strength of the internet connection; video quality for recorded dyadic interactions was crucial for the behavioral coding component. All instructions were provided to parents in English or Spanish based on their preference.</p> <hd id="AN0191574884-18">The Brief Early Relational Assessment (B-ERA)</hd> <p>The B-ERA (Clark, [<reflink idref="bib4" id="ref35">4</reflink>]) measures the quality of affect and behavior in parent–child interactions. Behavioral ratings from a 5-min videotaped free play are used to capture the child's experience of the parent, the parent's experience of the child, the affective and behavioral characteristics they each bring to the interaction, and the quality or tone of their relationship. The B-ERA was developed to classify both clinical and research observations and assess in a systematic manner the behavioral style and affective quality believed to be indicators of disturbed relationship patterns.</p> <hd id="AN0191574884-19">Adaptation to the B-ERA</hd> <p>The B-ERA consists of 5-min of free play between mom and child. A free-play activity is included in the DB-DOS. Researchers were able to use the same free play for both assessments to minimize participant burden. No additional changes were needed specific to the B-ERA for virtual adaptation.</p> <hd id="AN0191574884-20">Video Coding Training and Maintenance of Inter-Rater Reliability</hd> <p>Three coding systems were used for assessing parent and child outcomes. The expert video coder (EA) worked with three independent teams of three lab members: one team trained on B-ERA, a second team trained on the DB-DOS, and the third on the Parenting Clinical Observation (P-COS) (Hill et al., [<reflink idref="bib11" id="ref36">11</reflink>]). Each team was headed by a lead coder selected based on expertise and prior coding experience. The P-COS and DB-DOS teams trained with the expert coder for 3 weeks and the B-ERA team trained with her for 7 weeks, at which point the lead coder reached inter-rater reliability (IRR) with the expert coder on the respective manual. IRR was reached via 80% exact agreement on each code. The team leads continued training their groups until each team member achieved IRR with their lead coder. Coders were then assigned recordings, 20% of which were randomly selected for double coding and reviewed in consensus meetings with the lead coder to maintain IRR. Percent agreement was high across all coding systems. The mean exact agreement for the B-ERA was 78%. The DB-DOS had a range of 79–100% for individual items with a mean of 92% across all items. The P-COS had a range of 76–98% for individual items with a mean of 88% across all items. This training process was like the typical coding process of in-person observations, but more ongoing oversight was needed as well as additional double coding of some videos due to variability in video quality and occurrence of some non-standardized events (e.g., siblings or pets joining, and phone ringing).</p> <p>The expert coder consulted with each team on a biweekly basis to discuss questions or coding challenges and reviewed all recordings identified by the coding teams as having poor audio and/or video quality. In total, 298 of the 311 videos were coded (9 videos did not include the DB-DOS due to scheduling issues, 3 videos were missing due to storage malfunctions, and 1 video was deemed not codable by the expert coder due to quality). Seven poor-quality recordings were double-coded by the team leads to ensure accuracy.</p> <hd id="AN0191574884-21">Results</hd> <p></p> <hd id="AN0191574884-22">Lessons Learned Related to the Adapted Assessment Tools</hd> <p>Unanticipated challenges arose in the virtual adaptation of each assessment. In the AKT, PowerPoints loaded very slowly due to the video skit file sizes. To avoid wait time, assessors learned to open the PowerPoints ahead of the visit so that they were loaded when they began the AKT. Like the large AKT PowerPoint files, the scanned BSRA-3 stimulus PDF book was slow to load. To combat this, multiple PDF files were made for each subset so that they could be easily opened and navigated. Furthermore, though Zoom annotations were supported by many devices, they were sometimes disabled by assessors due to child distraction. Children would draw freely and not attend to prompts or become distracted by the color and annotation style options. In some cases, annotations had to be frequently enabled again after children were accidentally disabled while annotating.</p> <p>The MEFS was the most difficult activity for children, as they were often frustrated with the inability to sort the cards themselves. To combat this, assessors let children know before the MEFS administration that this would be a "game" in which they would work together as a team. To secure a "win" it was very important for children to point at the cards to help the assessor, who would move them to the correct box.</p> <p>The ideal remote DB-DOS/B-ERA setup was determined after practice in different locations (on a table, floor, bed, etc.) utilizing multiple devices. Assessors experimented with propping the devices and screen angles based on dyadic placement variations. This practice was useful in guiding parents during administration. Sibling interference also required additional guidance from assessors. Though siblings may have been present in other assessments, parents were able to help manage their interference as they had a better understanding of permissible feedback. However, siblings were attracted to the DB-DOS materials that included fun toys, which were difficult to manage during remote observations. During in-person administration, assessors can provide siblings with other materials and directly interact with them, so they do not interfere with the dyadic interaction. However, this was not possible during remote administration. To combat this, assessors inquired about siblings during scheduling and sent extra materials for siblings during this assessment. Parents were encouraged to direct all their attention to the study child as much as possible.</p> <hd id="AN0191574884-23">Lessons Learned Related to Adaptation and Implementation of Assessment Procedures</hd> <p></p> <hd id="AN0191574884-24">Technology</hd> <p>Participants had varying internet connection strengths. Due to the length of the visit and the data load with video and file sharing, some participants experienced issues with their Internet's bandwidth as indicated by Zoom warnings. When this occurred, participants were asked to disconnect and re-connect to the Zoom call. If possible, participants were asked to switch their data carrier (i.e., from their home Wi-Fi to cellular data). When these solutions were not effective, assessors offered to reschedule and complete the observation at another time. If this was not possible, participants continued to rejoin the call to complete the visit as best they could. The use of smartphones also proved challenging, as smaller screen displays made viewing assessment materials more difficult at times. Notifications temporarily blocked shared content and occasionally distracted from the completion of assessments. Incoming phone calls temporarily disabled device cameras.</p> <hd id="AN0191574884-25">Scheduling</hd> <p>Remote flexibility allowed assessors to accommodate participants' scheduling needs. For example, observations could be rescheduled more easily than if they were in-person, and participants were able to complete the assessments in more than one session to minimize the time burden. The increased flexibility afforded to participants may have increased adherence and resulted in a more representative sample. Participants could continue study involvement even when moving out of state, suggesting that remote observations have the potential to minimize study attrition.</p> <hd id="AN0191574884-26">Parent Role as Facilitator</hd> <p>For the successful completion of each assessment, parents' understanding of their facilitation role was crucial. Though assessors controlled the shared assessment content and prompts, parents had to actively ensure their children were attending to the content and assist in relaying their responses when needed. Shy children who initially did not want to engage with the assessors were often inaudible as they spoke softly or turned away from the screen. Parents had to encourage their children to speak up louder or relay whispered responses without repeating or rephrasing prompts. This varies considerably from an in-person assessment where the examiner could play a larger role in facilitation. The "parent cheat sheet" mentioned above was extremely useful in assisting parents in their facilitation role. The feedback survey findings indicate that parents understood their role to support their children's participation and understood when to provide feedback. Parents also indicated that instructions and materials were clear and useful. Recommendations for future remote observational studies include providing device-specific guidance with visuals and explicit "do/don't" language for each assessment.</p> <hd id="AN0191574884-27">Maintaining Child Engagement</hd> <p>Maintaining child engagement between assessments was very important given the number of assessments conducted. The use of the puppets from the AKT assessment was useful for building rapport with children. After the initial setup with parents at the beginning of each visit, assessors introduced themselves to children with the puppets. The assessors provided children with a developmentally appropriate overview of the "games" that they would play together with the puppets, followed by a welcome song. The puppets appeared in the AKT videos but were used throughout the call during transitions and breaks to encourage participation and provide developmentally appropriate engagement. Other useful engagement strategies included stretching and snack breaks, as well as time away from the screen in which the assessor turned off their Zoom camera until the dyad was ready to continue. Feedback survey findings suggest that these strategies were useful for children to feel comfortable engaging with the assessor and overall had a positive experience.</p> <hd id="AN0191574884-28">Cost Considerations</hd> <p>Since assessors did not travel to participants' homes, additional costs for remote work include materials sent to each participant and shipping expenses. To assure that participants completed their visits in their eligibility window, visit boxes were sent as soon as visits were scheduled and included materials (i.e., crayons coloring books, and free play toys) used during assessments. Some boxes were lost by FedEx or participants did not receive them (though marked delivered by FedEx), so additional boxes were sent to given participants. Compensation to offset data usage was another cost specific to remote administration.</p> <p>Although remote observations necessitated the shipping and materials costs described above, conducting observations remotely also alleviates research costs related to assessor travel and technology needs. Prior to the COVID-19 pandemic, assessors were scheduled to travel to participants' homes requiring gas mileage and commute time compensation. Furthermore, lodging would have been an additional cost for visits that required travel to regions of the state that were not within a short drive of the research team. Remote observations allowed assessors to complete as many visits as possible while not requiring them to leave their workplace. Travel time and costs for participants traveling to research labs if they did not feel comfortable with assessors in their homes were also eliminated. Zoom allowed a low-cost option for recording the observations. For in-person assessments, video cameras and storage devices would need to be purchased.</p> <hd id="AN0191574884-29">Feedback Survey Results</hd> <p>Of the 33 participants completing surveys, 11 had just completed the 48-month assessment, and 22 had completed the 54-month time point. The highest rating in the "ease of participation" domain was related to overall ease in participation (94% indicated agree or strongly agree). The highest rating in the technology domain was related to the ability for children to clearly see the Zoom content on their screens (100% agree or strongly agree). The highest rating in the visit's demands domain was related to the clarity and utilization of parent instructions and materials (97% indicated they agree or strongly agree). The highest rating in the child engagement domain was related to children's positive experience (94% agree or strongly agree). Only 36% of participants agreed or strongly agreed that their child would have performed differently in an in-person visit. Among participants who indicated their children would have performed differently in-person, five said their child would have performed a lot better in-person and six said their child would have performed a little better. When asked about their preference for remote versus in-person observations, the vast majority (61.29%) indicated a preference for virtual, while only 16.13% preferred in-person at their homes, 3.23% preferred in-person at a university or community office, and 19.35% indicated no preference.</p> <hd id="AN0191574884-30">Devices Use and Technology Familiarity Results</hd> <p>The Device Use and Familiarity Survey was completed by assessors for all 311 completed observations. The most used devices across all three timepoints were smartphones (67.20%), followed by laptop computers (18.01%), and then tablets (10.93%). Chromebook use (3.54%) was distinguished from general laptop computers since they had limited functionality. The least used device was a desktop computer (<emph>N</emph> = 1, 0.32%).</p> <p>Children had varying previous experiences with the device used (or a similar one) within the last 3 months of completing the visit. Some children used the device (or a similar one) every day (41.48%), or a few times a week (17.68%) in the last 3 months. However, some children never (17.68%) or rarely (16.40%) used the device (or a similar one) in the last 3 months. 6.75% of children used the device once a week in the last 3 months.</p> <p>Parents reported variability in children's exposure to video calls on any platform within the last 3 months of completing the visit (Zoom, Skype, Facetime, Google Hangouts, etc.). Some children participated in daily video calls (20.90%) or at least once per week (24.44%). However, some children never participated in video calls (15.11) or only once or twice (18.33) in the last 3 months. 21.22% of children occasionally participated in video calls (3 or more times but not regularly) (see Supplemental Material for copy of survey).</p> <hd id="AN0191574884-31">Discussion</hd> <p>The disruption of the COVID-19 pandemic forced a rapid reimagining of research protocols that pivoted from in-person to remote study procedures. While most research employing remote observational methods has been conducted with adults, this study adds to the growing literature describing the use of virtual observational methods with children (e.g., García-Álvarez et al., [<reflink idref="bib9" id="ref37">9</reflink>]; Gijbels et al., [<reflink idref="bib10" id="ref38">10</reflink>]; Krogh-Jespersen et al., [<reflink idref="bib13" id="ref39">13</reflink>]; Manning et al., [<reflink idref="bib14" id="ref40">14</reflink>]; McElwain et al., [<reflink idref="bib15" id="ref41">15</reflink>]; Schein et al., [<reflink idref="bib22" id="ref42">22</reflink>]; Shin et al., [<reflink idref="bib25" id="ref43">25</reflink>]; Segal &amp; Moulson, [<reflink idref="bib23" id="ref44">23</reflink>]). Our team successfully modified assessment protocols and assessment tools for virtual administration and completed multiple assessments at each observational timepoint to assess various parenting and child self-regulation domains. Based on the success of our virtual observations, we plan to continue their use in future prevention intervention trials and believe that the details provided in this manuscript can also guide other research teams conducting virtual observations with children and their caregivers.</p> <p>A central tenet of the shift from in-person to remote research is a commitment to accessibility. Just as the popularizing of telehealth services has expanded access to clinical care, remote research may have the same effect. Several recent studies have highlighted the opportunity for research to be more inclusive of racially and socioeconomically diverse samples by limiting barriers to participation via provision of remote options (Krogh-Jespersen et al., [<reflink idref="bib13" id="ref45">13</reflink>]; McElwain et al., [<reflink idref="bib15" id="ref46">15</reflink>]; Rhodes et al., [<reflink idref="bib21" id="ref47">21</reflink>]; Segal &amp; Moulson, [<reflink idref="bib23" id="ref48">23</reflink>]). Our study, like many other preventive intervention trials, aimed to collect data longitudinally from participants, requiring multiple data collection timepoints. Many barriers have been identified to conducting in-person assessments with racially and socioeconomically diverse caregivers and their children, including limited financial resources to travel to assessment locations, difficulties finding and paying for childcare for other children in the household, residential mobility, and the long blocks of time needed to conduct observations. The transition to virtual observations minimized these barriers for our racially and socioeconomically diverse sample and allowed our team to conduct over 300 observations among both English- and Spanish-speaking families.</p> <p>Virtual assessments provide novel logistical challenges, however. Families used various devices to participate in assessments, including smartphones that had small screens to view assessment materials. Costs associated with data/internet usage are also a potential barrier, with our study providing stipends to participants to offset these costs. Materials for observations also needed to be sent to participants in advance which required coordination between the research team and participants to prevent materials being lost or stolen during the delivery process. The ability to conduct assessments in the comfort of one's home, while minimizing transportation barriers, creates separate challenges including the need to minimize disruptions in the household, as well as on one's phone. Additionally, remote assessments required the research team to provide detailed instructions to caregivers on how to engage in each observational assessment. In pivoting to virtual assessments to measure infant temperament and mother-infant interactions, Shin et al. ([<reflink idref="bib25" id="ref49">25</reflink>]) asked participants to set smartphones to "do not disturb," provided recommendations for dyad positioning, and developed printed visual instructions for assessments that were shared with participants when they needed additional guidance. Although our procedures were effective in producing high-quality videos for coding, the use of additional strategies like those used by Shin et al. may further minimize potential disruptions to virtual assessments. That said, some potential procedures to minimize disruption were deemed inappropriate by our research team. For example, our sample consisted of many caregivers with older children who did not have a spouse or partner and utilized phone communication for childcare arrangements and updates; therefore, we did not ask participants to enable "do not disturb" settings on their phones in the event there were emergencies with their other children.</p> <p>Our study concurs with recommendations of other research teams that pivoted to remote observation that assessors require additional training to address unique challenges of remote administration, and additional time needs to be spent prepping families prior to conducting assessments. Several studies (Gijbels et al., [<reflink idref="bib10" id="ref50">10</reflink>]; Krogh-Jespersen et al., [<reflink idref="bib13" id="ref51">13</reflink>]; Rhodes et al., [<reflink idref="bib21" id="ref52">21</reflink>]; Shin et al., [<reflink idref="bib25" id="ref53">25</reflink>]) highlight the importance of providing clear instructions to parents upfront, and throughout the assessment, to help them understand their role and limit interference in children's responses. Additionally, our study affirms recommendations from other studies conducting virtual parent and child assessments that children must be engaged in different ways by providing more frequent breaks, entertaining them throughout, and motivating them to participate through encouragement and making activities into a "game" (Gijbels et al., [<reflink idref="bib10" id="ref54">10</reflink>]; Rhodes et al., [<reflink idref="bib21" id="ref55">21</reflink>]). We further recommend that assessors receive training on how to work with diverse families who experience different living situations—some more precarious than others—in ways that are culturally sensitive and non-judgmental. As witnessed in our study, families' living situations vary greatly, and assessors must be able to think on their feet and sensitively and respectfully work with families to provide optimal conditions for remote assessments, even when aspects of the home environment are not most conducive.</p> <hd id="AN0191574884-32">Strengths and Limitations</hd> <p>This study contributes to a rapidly growing literature on conducting virtual observations with caregivers and their children. A particular strength of this study was engagement with a predominately low socioeconomic sample, of whom many conducted assessments in Spanish. Since many previous studies pivoting to virtual observations during the COVID-19 pandemic did not work with racially, ethnically, or socioeconomically diverse samples, our study provides useful insights that heighten generalizability of procedures and lessons learned from virtual assessments to a more diverse sample of caregivers and children. We found that for the under-resourced families in our sample, approaches used by our research team allowed them to participate meaningfully in remote assessments despite potential barriers such as poor internet connectivity, variation in devices on which assessments were conducted, and geographic distance between them and the assessor. Collecting caregiver survey data on their virtual observations extends existing literature that largely focuses on only presenting research team perspectives. Our caregivers indicated high satisfaction with virtual observation procedures, including the pace of activities, expectations for the virtual sessions, and their child's comfort level. Importantly, caregivers also found it convenient to participate in virtual assessments, with most indicating a preference for virtual assessments if given a choice between virtual, in-home, or off-site administration. We also reported on the quality of the video observations, noting that only one observation had to be removed due to poor data quality. Moreover, data quality was sufficiently robust that independent coders were able to generate inter-rater reliability of greater than 75% for all assessments, which are comparable to reliability levels when coding in-person administrations of the same assessments (O'Dor et al., [<reflink idref="bib17" id="ref56">17</reflink>]; Else-Quest et al., [<reflink idref="bib8" id="ref57">8</reflink>]).</p> <p>This study is not without limitations, however. There are potential limitations on the generalizability of study findings, as a slightly higher percentage of individuals who were married and were not receiving mental health services completed at least one study assessment. Although we collected caregiver perspectives on virtual observations, this was done for a subsample and a larger sample would have allowed us to look at any emerging patterns within subgroups (e.g., Spanish-speaking families, caregivers with other children in the home). Additionally, although we attempted to conduct observations at three time points with participants, we only conducted the Caregiver Feedback Survey after one observation. It is possible that earlier observations were more challenging because of the novelty of the remote setup, with assessors noting this anecdotally, but we did not conduct multiple surveys with participants over time to document potential trends. Many companies that license parenting and child observational assessments have now created digital versions given requests by research teams such as ours to conduct virtual assessments during the COVID-19 pandemic. Thus, some of our specific approaches related to adapting assessments for virtual use may be less necessary as assessment developers create new virtual formats. However, this manuscript still provides valuable, detailed guidance on procedures that prepared caregivers and children for virtual assessments and conducting the virtual assessments with them. Finally, this study did not compare virtual and in-person observations, which limits our ability to recommend virtual observations as the preferred mode for conducting observations. Future research could compare virtual and in-person observations to ascertain whether differences exist in the ability to administer and code different types of observations. Subsequent studies could also allow families to choose their preference of virtual or in-person observations to determine whether preference in administration modality may affect engagement with the data collection procedures and/or the quality of data obtained.</p> <hd id="AN0191574884-33">Conclusions</hd> <p>Many preventive interventions use observational measures to evaluate their effectiveness because they are considered objective assessments of behavior that are less subject to bias. Many of these preventive interventions, including MB, focus on improving outcomes for young children and their parents which makes it essential to be able to conduct high-quality observations. This study demonstrated that we could effectively and reliably conduct and code remote parent and child assessments, providing other research teams confidence and guidance to make similar pivots to remote administration. We also provide specific guidance on key procedural considerations related to instrument adaptation, remote data collection procedures, and technology support. Just as we prioritized the development of clear, step-by-step instructions to help parents navigate the novelty of virtual observations, we hope this paper serves as a thorough blueprint for researchers as they embark on the transition from in-person to remote assessments.</p> <hd id="AN0191574884-34">Author Contribution</hd> <p>SDT is the principal investigator of the study and was the principal investigator on the parent study. He oversaw the conceptualization, planning, and execution of the study and drafted/edited the manuscript. JC is a project coordinator on the study who assisted in data collection, conducted observational assessments, was involved in video coding, and drafted/edited the manuscript. AD is the data manager on the study and served in a similar role on the parent study. She was involved in the conceptualization, planning, and execution of the study, and drafted/edited the manuscript. AM is a doctoral student in clinical psychology who was involved in video coding and drafting/editing the manuscript. KA and ZW are research study coordinators on the study who conducted observational assessments and edited the manuscript. AL and EM are research study coordinators on the study who assisted in data collection and drafting/editing of the manuscript. KA, AL, and EM were involved in video coding. MB was a project coordinator on the study who coordinated the transition to virtual observational assessments and edited the manuscript. EA served as a child development video coding assessment expert on the study and edited the manuscript. LSW is a co-investigator of the study, was involved in the planning and execution of the study and edited the manuscript.</p> <hd id="AN0191574884-35">Funding</hd> <p>This work was supported by the Eunice Kennedy Shriver National Institute of Child Health &amp; Human Development of the National Institutes of Health (RO1HD097215).</p> <hd id="AN0191574884-36">Data Availability</hd> <p>All data from this study are available via reasonable request to corresponding author.</p> <hd id="AN0191574884-37">Declarations</hd> <p></p> <hd id="AN0191574884-38">Ethics Approval</hd> <p>This study has been approved by the Institutional Review Board (IRB) of Northwestern University (NU) STU00209764. All research procedures were conducted in accordance with the ethical standards of the IRB and with the principles of the 1964 Helsinki Declaration and its later amendments.</p> <hd id="AN0191574884-39">Consent to Participate</hd> <p>Informed consent was obtained from all participants in the study. A waiver of assent for the children participating in the observational assessments was approved by the IRB at NU.</p> <hd id="AN0191574884-40">Competing Interests</hd> <p>The authors declare no competing interests.</p> <hd id="AN0191574884-41">Supplementary Information</hd> <p>Below is the link to the electronic supplementary material.</p> <p>Graph: Supplementary file1 (PDF 168 KB)</p> <p>Graph: Supplementary file2 (PDF 119 KB)</p> <p>Graph: Supplementary file3 (PDF 175 KB)</p> <hd id="AN0191574884-42">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0191574884-43"> <title> References </title> <blist> <bibl id="bib1" idref="ref12" type="bt">1</bibl> <bibtext> Alhusen JL, Hayat MJ, Borg L. A pilot study of a group-based perinatal depression intervention on reducing depressive symptoms and improving maternal-fetal attachment and maternal sensitivity. Archives of Women's Mental Health. 2021; 24; 1: 145-154. 10.1007/s00737-020-01032-0. 32409986</bibtext> </blist> <blist> <bibl id="bib2" idref="ref13" type="bt">2</bibl> <bibtext> Bracken BA, Panter JE. Using the Bracken Basic Concept Scale and Bracken Concept Development Program in the assessment and remediation of young children's concept development: Basic Concept Assessment and Intervention. Psychology in the Schools. 2011; 48; 5: 464-475. 10.1002/pits.20568</bibtext> </blist> <blist> <bibl id="bib3" idref="ref14" type="bt">3</bibl> <bibtext> Bruce A. B. (2024). Bracken basic concept scale, 4th edn. Pearson Assessments. https://<ulink href="http://www.pearsonassessments.com/store/usassessments/en/Store/Professional-Assessments/Cognition-%26-Neuro/Bracken-Basic-Concept-Scale-%7C-Fourth-Edition%3A-Receptive/p/P100030000.html">www.pearsonassessments.com/store/usassessments/en/Store/Professional-Assessments/Cognition-%26-Neuro/Bracken-Basic-Concept-Scale-%7C-Fourth-Edition%3A-Receptive/p/P100030000.html</ulink></bibtext> </blist> <blist> <bibl id="bib4" idref="ref35" type="bt">4</bibl> <bibtext> Clark R. The brief parent-Child early relational assessment [Handbook]. 2019; Department of Psychiatry, University of Wisconsin Medical School</bibtext> </blist> <blist> <bibl id="bib5" idref="ref27" type="bt">5</bibl> <bibtext> Denham SA, Bassett HH, Brown C, Way E, Steed J. "I know how you feel": Preschoolers' emotion knowledge contributes to early school success. Journal of Early Childhood Research: ECR. 2015; 13; 3: 252-262. 10.1177/1476718X13497354</bibtext> </blist> <blist> <bibl id="bib6" idref="ref28" type="bt">6</bibl> <bibtext> Denham SA, Bassett HH, Zinsser KM, Bradburn IS, Bailey CS, Shewark EA, Ferrier DE, Liverette KH, Steed J, Karalus SP, Kianpour S. Computerized social-emotional assessment measures for early childhood settings. Early Childhood Research Quarterly. 2020; 51: 55-66. 10.1016/j.ecresq.2019.07.002</bibtext> </blist> <blist> <bibl id="bib7" idref="ref19" type="bt">7</bibl> <bibtext> Diebold A, Johnson JK, Brennan M, Ciolino JD, Petitclerc A, Wakschlag LS, Garfield CF, Yeh C, Lovejoy A, Zakieh D, Tandon SD. Testing the effects of a prenatal depression preventive intervention on parenting and young children's self-regulation and functioning (EPIC): Protocol for a longitudinal observational study. BMC Public Health. 2021; 21; 1: 1368. 10.1186/s12889-021-11385-5. 34246233. 8271322</bibtext> </blist> <blist> <bibl id="bib8" idref="ref57" type="bt">8</bibl> <bibtext> Else-Quest NM, Clark R, Owen MT. Stability in mother-child interactions from infancy through adolescence. Parenting, Science and Practice. 2011; 11; 4: 280-287. 10.1080/15295192.2011.613724. 22685382. 3368597</bibtext> </blist> <blist> <bibl id="bib9" idref="ref4" type="bt">9</bibl> <bibtext> García-Álvarez D, Soler MJ, Achard-Braga L. Psychological well-being in teachers during and post-COVID-19: Positive psychology interventions. Frontiers in Psychology. 2021; 12: 769363. 10.3389/fpsyg.2021.769363. 34975659. 8716601</bibtext> </blist> <blist> <bibtext> Gijbels L, Cai R, Donnelly PM, Kuhl PK. Designing virtual, moderated studies of early childhood development. Frontiers in Psychology. 2021; 12: 740290. 10.3389/fpsyg.2021.740290. 34707545. 8542922</bibtext> </blist> <blist> <bibtext> Hill C, Maskowitz K, Danis B, Wakschlag L. Validation of a clinically sensitive, observational coding system for parenting behaviors: The parenting clinical observation schedule. Parenting. 2008; 8; 2: 153-185. 10.1080/15295190802045469. 37124424. 10134773</bibtext> </blist> <blist> <bibtext> Jensen JK, Ciolino JD, Diebold A, Segovia M, Degillio A, Solano-Martinez J, Tandon SD. Comparing the effectiveness of clinicians and paraprofessionals to reduce disparities in perinatal depression via the mothers and babies course: Protocol for a cluster-randomized controlled trial. JMIR Research Protocols. 2018; 7; 11: e11624. 10.2196/11624. 30459138. 6280028</bibtext> </blist> <blist> <bibtext> Krogh-Jespersen S, MacNeill LA, Anderson EL, Stroup HE, Harriott EM, Gut E, Blum A, Fareedi E, Fredian KM, Wert SL, Wakschlag LS, Norton ES. Disruption leads to methodological and analytic innovation in developmental sciences: Recommendations for remote administration and dealing with messy data. Frontiers in Psychology. 2022; 12: 732312. 10.3389/fpsyg.2021.732312. 35058833. 8764157</bibtext> </blist> <blist> <bibtext> Manning BL, Harpole A, Harriott EM, Postolowicz K, Norton ES. Taking language samples home: Feasibility, reliability, and validity of child language samples conducted remotely with video chat versus in-person. Journal of Speech, Language, and Hearing Research. 2020; 63; 12: 3982-3990. 10.1044/2020_JSLHR-20-00202. 33186507. 8608210</bibtext> </blist> <blist> <bibtext> McElwain NL, Hu Y, Li X, Fisher MC, Baldwin JC, Bodway JM. Zoom, zoom, baby! Assessing mother-infant interaction during the still face paradigm and infant language development via a virtual visit procedure. Frontiers in Psychology. 2022; 12: 734492. 10.3389/fpsyg.2021.734492. 35250686. 8888453</bibtext> </blist> <blist> <bibtext> McFarlane E, Burrell L, Duggan A, Tandon D. Outcomes of a randomized trial of a cognitive behavioral enhancement to address maternal distress in home visited mothers. Maternal and Child Health Journal. 2017; 21; 3: 475-484. 10.1007/s10995-016-2125-7. 27535131. 5357278</bibtext> </blist> <blist> <bibtext> O'Dor SL, Grasso DJ, Forbes D, Bates JE, McCarthy KJ, Wakschlag LS, Briggs-Gowan MJ. The family socialization interview-revised (FSI-R): A comprehensive assessment of parental disciplinary behaviors. Prevention Science: The Official Journal of the Society for Prevention Research. 2017; 18; 3: 292-304. 10.1007/s11121-016-0707-7. 27718104. 5568657</bibtext> </blist> <blist> <bibtext> Pettman D, O'Mahen H, Blomberg O, Svanberg AS, von Essen L, Woodford J. Effectiveness of cognitive behavioural therapy-based interventions for maternal perinatal depression: A systematic review and meta-analysis. BMC Psychiatry. 2023; 23; 1: 208-208. 10.1186/s12888-023-04547-9. 36991389. 10052839</bibtext> </blist> <blist> <bibtext> Pew Research Center. Internet/Broadband Fact Sheet. 2020; Pew Research Center</bibtext> </blist> <blist> <bibtext> Reflection Sciences (2024). Minnesota executive function scale. Reflection Sciences. https://reflectionsciences.com/mefs/. Accessed 1 Oct 2024.</bibtext> </blist> <blist> <bibtext> Rhodes M, Rizzo MT, Foster-Hanson E, Moty K, Leshin RA, Wang M, Benitez J, Ocampo JD. Advancing developmental science via unmoderated remote research with children. Journal of Cognition and Development. 2020; 21; 4: 477-493. 10.1080/15248372.2020.1797751. 32982602. 7513948</bibtext> </blist> <blist> <bibtext> Schein SS, Roben CKP, Costello AH, Dozier M. Assessing changes in parent sensitivity in telehealth and hybrid implementation of attachment and biobehavioral catch-up during the COVID-19 pandemic. Child Maltreatment. 2023; 28; 1: 24-33. 10.1177/10775595211072516. 35081800</bibtext> </blist> <blist> <bibtext> Segal SC, Moulson MC. Parent-infant interaction tasks adapted for remote testing: Strengths, challenges, and recommendations. Frontiers in Psychology. 2021; 12: 733275. 10.3389/fpsyg.2021.733275. 34721201. 8548735</bibtext> </blist> <blist> <bibtext> Shields MM, McGinnis MN, Selmeczy D. Remote research methods: Considerations for work with children. Frontiers in Psychology. 2021; 12: 703706. 10.3389/fpsyg.2021.703706. 34777090. 8581245</bibtext> </blist> <blist> <bibtext> Shin E, Smith CL, Howell BR. Advances in behavioral remote data collection in the home setting: Assessing the mother-infant relationship and infant's adaptive behavior via virtual visits. Frontiers in Psychology. 2021; 12: 703822. 10.3389/fpsyg.2021.703822. 34659017. 8517484</bibtext> </blist> <blist> <bibtext> US Preventive Services Task ForceCurry SJ, Krist AH, Owens DK, Barry MJ, Caughey AB, Davidson KW, Doubeni CA, Epling JW Jr, Grossman DC, Kemper AR, Kubik M, Landefeld CS, Mangione CM, Silverstein M, Simon MA, Tseng CW, Wong JB. Interventions to prevent perinatal depression: US preventive services task force recommendation statement. JAMA. 2019; 321; 6: 580-587. 10.1001/jama.2019.0007</bibtext> </blist> <blist> <bibtext> Wakschlag LS, Hill C, Carter AS, Danis B, Egger HL, Keenan K, Leventhal BL, Cicchetti D, Maskowitz K, Burns J, Briggs-Gowan MJ. Observational assessment of preschool disruptive behavior, Part I: Reliability of the disruptive behavior diagnostic observation schedule (DB-DOS). Journal of the American Academy of Child and Adolescent Psychiatry. 2008; 47; 6: 622-631. 18434926. 10159266</bibtext> </blist> <blist> <bibtext> Zaadnoordijk, L. &amp; Cusack, R. (2022). Online testing in developmental science: A guide to design and implementation. In R. O. Gilmore &amp; J. J. Lockman (Eds.), Advances in child development and behavior: New methods and approaches for studying child development (Vol. 62, pp. 93–125). Academic Press.</bibtext> </blist> </ref> <aug> <p>By S. Darius Tandon; Jocelyne Chavez; Alicia Diebold; Ada Moses; Aiko E. Lovejoy; Zechao Wang; Katerine Arevalo; Elaine McBride; Marianne Brennan; Erica Anderson and Lauren S. Wakschlag</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib24" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib28" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib19" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib10" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib13" firstref="ref6"></nolink> <nolink nlid="nl6" bibid="bib14" firstref="ref7"></nolink> <nolink nlid="nl7" bibid="bib15" firstref="ref8"></nolink> <nolink nlid="nl8" bibid="bib22" firstref="ref9"></nolink> <nolink nlid="nl9" bibid="bib25" firstref="ref10"></nolink> <nolink nlid="nl10" bibid="bib23" firstref="ref11"></nolink> <nolink nlid="nl11" bibid="bib26" firstref="ref15"></nolink> <nolink nlid="nl12" bibid="bib18" firstref="ref16"></nolink> <nolink nlid="nl13" bibid="bib16" firstref="ref18"></nolink> <nolink nlid="nl14" bibid="bib12" firstref="ref20"></nolink> <nolink nlid="nl15" bibid="bib42" firstref="ref26"></nolink> <nolink nlid="nl16" bibid="bib20" firstref="ref29"></nolink> <nolink nlid="nl17" bibid="bib27" firstref="ref32"></nolink> <nolink nlid="nl18" bibid="bib11" firstref="ref36"></nolink> <nolink nlid="nl19" bibid="bib21" firstref="ref47"></nolink> <nolink nlid="nl20" bibid="bib17" firstref="ref56"></nolink> |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: EJ1508200 AccessLevel: 3 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Zoom to the Virtual Room: The Shift to Remote Early Childhood Observational Assessments – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22S%2E+Darius+Tandon%22">S. Darius Tandon</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-3243-903X">0000-0002-3243-903X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Jocelyne+Chavez%22">Jocelyne Chavez</searchLink><br /><searchLink fieldCode="AR" term="%22Alicia+Diebold%22">Alicia Diebold</searchLink><br /><searchLink fieldCode="AR" term="%22Ada+Moses%22">Ada Moses</searchLink><br /><searchLink fieldCode="AR" term="%22Aiko+E%2E+Lovejoy%22">Aiko E. Lovejoy</searchLink><br /><searchLink fieldCode="AR" term="%22Zechao+Wang%22">Zechao Wang</searchLink><br /><searchLink fieldCode="AR" term="%22Katerine+Arevalo%22">Katerine Arevalo</searchLink><br /><searchLink fieldCode="AR" term="%22Elaine+McBride%22">Elaine McBride</searchLink><br /><searchLink fieldCode="AR" term="%22Marianne+Brennan%22">Marianne Brennan</searchLink><br /><searchLink fieldCode="AR" term="%22Erica+Anderson%22">Erica Anderson</searchLink><br /><searchLink fieldCode="AR" term="%22Lauren+S%2E+Wakschlag%22">Lauren S. Wakschlag</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="SO" term="%22Prevention+Science%22"><i>Prevention Science</i></searchLink>. 2026 27(1):28-40. – Name: Avail Label: Availability Group: Avail Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: Y – Name: Pages Label: Page Count Group: Src Data: 13 – Name: DatePubCY Label: Publication Date Group: Date Data: 2026 – Name: SourceSuprt Label: Sponsoring Agency Group: SrcSuprt Data: Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) (DHHS/NIH) – Name: NumberContract Label: Contract Number Group: NumCntrct Data: RO1HD097215 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Journal Articles<br />Reports - Research – Name: Audience Label: Education Level Group: Audnce Data: <searchLink fieldCode="EL" term="%22Early+Childhood+Education%22">Early Childhood Education</searchLink> – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22Early+Childhood+Education%22">Early Childhood Education</searchLink><br /><searchLink fieldCode="DE" term="%22Observation%22">Observation</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Distance+Education%22">Distance Education</searchLink><br /><searchLink fieldCode="DE" term="%22Videoconferencing%22">Videoconferencing</searchLink><br /><searchLink fieldCode="DE" term="%22Virtual+Classrooms%22">Virtual Classrooms</searchLink><br /><searchLink fieldCode="DE" term="%22COVID-19%22">COVID-19</searchLink><br /><searchLink fieldCode="DE" term="%22Pandemics%22">Pandemics</searchLink><br /><searchLink fieldCode="DE" term="%22Children%22">Children</searchLink><br /><searchLink fieldCode="DE" term="%22Child+Caregivers%22">Child Caregivers</searchLink><br /><searchLink fieldCode="DE" term="%22Longitudinal+Studies%22">Longitudinal Studies</searchLink><br /><searchLink fieldCode="DE" term="%22Birth%22">Birth</searchLink><br /><searchLink fieldCode="DE" term="%22Mothers%22">Mothers</searchLink><br /><searchLink fieldCode="DE" term="%22Depression+%28Psychology%29%22">Depression (Psychology)</searchLink><br /><searchLink fieldCode="DE" term="%22Prevention%22">Prevention</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Parenting+Styles%22">Parenting Styles</searchLink><br /><searchLink fieldCode="DE" term="%22Low+Income+Groups%22">Low Income Groups</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Interrater+Reliability%22">Interrater Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Home+Visits%22">Home Visits</searchLink><br /><searchLink fieldCode="DE" term="%22Preferences%22">Preferences</searchLink> – Name: DOI Label: DOI Group: ID Data: 10.1007/s11121-024-01737-z – Name: ISSN Label: ISSN Group: ISSN Data: 1389-4986<br />1573-6695 – Name: Abstract Label: Abstract Group: Ab Data: The COVID-19 pandemic has led prevention researchers to increasingly use remote observational procedures given social distancing directives associated with the pandemic. However, few studies have used remote observational procedures with children and their caregivers, with scant literature describing procedures with socioeconomically, racially, and ethnically diverse families. This manuscript describes processes to pivot to remote assessment of parent and child observations in the context of a longitudinal study examining the effects of a postpartum depression preventive intervention on responsive parenting and child self-regulation. We conducted remote assessments across three timepoints--42, 48, and 54 months--with 133 low-income and racially and ethnically diverse parent--child dyads. Details are provided on remote observation preparation and setup, as well as adaptation of observational assessments. Lessons learned are shared on the use of technology, scheduling considerations, parent's role as facilitator, maintaining child engagement, and cost considerations. We demonstrated excellent inter-rater reliability between independent coders on all assessments, suggesting the quality of remote assessments was conducive for analysis. Surveys with families completing a remote assessment found that most felt it was easy to participate in remote assessments and their child had a positive experience. Most parents preferred virtual visits if given an option for future assessments. Much prevention research utilizes observational measures that are less subject to bias. Our study demonstrated that research teams can effectively and reliably pivot to remote assessment with racially and ethnically diverse, low socioeconomic families, thereby providing guidance to other prevention researchers considering similar remote assessments with diverse samples. – Name: AbstractInfo Label: Abstractor Group: Ab Data: As Provided – Name: DateEntry Label: Entry Date Group: Date Data: 2026 – Name: AN Label: Accession Number Group: ID Data: EJ1508200 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1508200 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s11121-024-01737-z Languages: – Text: English PhysicalDescription: Pagination: PageCount: 13 StartPage: 28 Subjects: – SubjectFull: Early Childhood Education Type: general – SubjectFull: Observation Type: general – SubjectFull: Evaluation Methods Type: general – SubjectFull: Distance Education Type: general – SubjectFull: Videoconferencing Type: general – SubjectFull: Virtual Classrooms Type: general – SubjectFull: COVID-19 Type: general – SubjectFull: Pandemics Type: general – SubjectFull: Children Type: general – SubjectFull: Child Caregivers Type: general – SubjectFull: Longitudinal Studies Type: general – SubjectFull: Birth Type: general – SubjectFull: Mothers Type: general – SubjectFull: Depression (Psychology) Type: general – SubjectFull: Prevention Type: general – SubjectFull: Intervention Type: general – SubjectFull: Parenting Styles Type: general – SubjectFull: Low Income Groups Type: general – SubjectFull: Technology Uses in Education Type: general – SubjectFull: Interrater Reliability Type: general – SubjectFull: Home Visits Type: general – SubjectFull: Preferences Type: general Titles: – TitleFull: Zoom to the Virtual Room: The Shift to Remote Early Childhood Observational Assessments Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: S. Darius Tandon – PersonEntity: Name: NameFull: Jocelyne Chavez – PersonEntity: Name: NameFull: Alicia Diebold – PersonEntity: Name: NameFull: Ada Moses – PersonEntity: Name: NameFull: Aiko E. Lovejoy – PersonEntity: Name: NameFull: Zechao Wang – PersonEntity: Name: NameFull: Katerine Arevalo – PersonEntity: Name: NameFull: Elaine McBride – PersonEntity: Name: NameFull: Marianne Brennan – PersonEntity: Name: NameFull: Erica Anderson – PersonEntity: Name: NameFull: Lauren S. Wakschlag IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 1389-4986 – Type: issn-electronic Value: 1573-6695 Numbering: – Type: volume Value: 27 – Type: issue Value: 1 Titles: – TitleFull: Prevention Science Type: main |
| ResultId | 1 |
