CoDAS
http://www.codas.periodikos.com.br/article/doi/10.1590/2317-1782/e20240224en
CoDAS
Artigo Original

A preliminary psychometric evaluation of the activity ordering task with a metacognitive facet (AOT-M)

Nidhi Lalu Jacob; Aysha Rooha; Anjaly S. Nair; Gagan Bajaj; Vinitha Mary George; Jayashree S. Bhat

Downloads: 0
Views: 20

Abstract

ABSTRACT: Purpose: The Activity Ordering Task with a metacognitive facet (AOT-M) was developed, in our previous work, to address the disconnect between traditional working memory (WM) tasks and everyday WM demands, the lack of culturally sensitive, context-based WM tasks in India and enhance participant engagement. The present study aims to provide preliminary evidence of the AOT-M's psychometric properties among a non-clinical adult population, evaluate its sensitivity to cognitive and metacognitive changes with aging, establish construct validity, ecological validity, concurrent validity and test-retest reliability.

Methods: Ninety neurotypical adults, evenly distributed across three age groups, participated in the study. Descriptive statistics examined the distribution of performance spans and estimation discrepancies across age groups and the age-related statistical differences were evaluated using the Kruskal-Wallis Test. Construct validity was assessed using Rasch analysis, while ecological validity was evaluated with the Multidimensional Assessment of Research in Context (MARC) tool. Concurrent validity with sentence ordering and digit letter ordering tasks, was determined through Pearson’s correlation coefficient and test-retest reliability was assessed using the Intraclass Correlation Coefficient and Bland-Altman plots.

Results: The patterns observed in WM performance spans and estimation discrepancies highlighted the task's sensitivity to aging related cognitive and metacognitive changes. Evidence from the MARC tool substantiated ecological validity, and concurrent validity was demonstrated through significant correlations with established WM tasks. While Rasch analysis supported construct validity, moderate person reliability indicated some limitations in task sensitivity. The AOT-M demonstrated good test-retest reliability.

Conclusion: Overall, the study provides preliminary evidence of the AOT-M’s good psychometric properties within a neurotypical adult sample, suggesting it to be a promising addition to the cognitive communicative toolbox for Speech Language Pathologists.

Keywords

Aging, Metacognition, Working Memory, Reliability, Validity

Referências

1 Muñoz-Pradas R, Díaz-Palacios M, Rodriguez-Martínez EI, Gómez CM. Order of maturation of the components of the working memory from childhood to emerging adulthood. Curr Res Behav Sci. 2021;2:100062. http://doi.org/10.1016/j.crbeha.2021.100062.

2 Brehmer Y, Westerberg H, Bäckman L. Working-memory training in younger and older adults: training gains, transfer, and maintenance. Front Hum Neurosci. 2012;6:63. http://doi.org/10.3389/fnhum.2012.00063. PMid:22470330.

3 Pliatsikas C, Veríssimo J, Babcock L, Pullman MY, Glei DA, Weinstein M, et al. Working memory in older adults declines with age, but is modulated by sex and education. Q J Exp Psychol. 2019;72(6):1308-27. http://doi.org/10.1177/1747021818791994. PMid:30012055.

4 Mohapatra B, Laures-Gore J. Moving toward accurate assessment of working memory in adults with neurogenically based communication disorders. Am J Speech Lang Pathol. 2021;30(3):1292-300. http://doi.org/10.1044/2021_AJSLP-20-00305. PMid:33970679.

5 Nigam R, Kar BR. Cognitive ageing in developing societies: an overview and a cross-sectional study on young, middle-aged and older adults in the Indian context. Psychol Dev Soc J. 2020;32(2):278-307. http://doi.org/10.1177/0971333620937511.

6 George VM, Bajaj G, Bhat JS. Efficacy of working memory training in middle-aged adults. Commun Sci Disord. 2020;25(4):830-56. http://doi.org/10.12963/csd.20768.

7 Redick TS, Broadway JM, Meier ME, Kuriakose PS, Unsworth N, Kane MJ, et al. Measuring working memory capacity with automated complex span tasks. Eur J Psychol Assess. 2012;28(3):164-71. http://doi.org/10.1027/1015-5759/a000123.

8 Carrigan N, Barkus E. A systematic review of cognitive failures in daily life: healthy populations. Neurosci Biobehav Rev. 2016;63:29-42. http://doi.org/10.1016/j.neubiorev.2016.01.010. PMid:26835660.

9 Brown KD, Schmitter-Edgecombe M. A clinic-based measure of everyday planning ability: the overnight trip task. Arch Clin Neuropsychol. 2024;39(1):51-64. http://doi.org/10.1093/arclin/acad052. PMid:37489707.

10 Okahashi S, Seki K, Nagano A, Luo Z, Kojima M, Futaki T. A virtual shopping test for realistic assessment of cognitive function. J Neuroeng Rehabil. 2013;10(1):59. http://doi.org/10.1186/1743-0003-10-59. PMid:23777412.

11 Sanz Simon S, Ben-Eliezer D, Pondikos M, Stern Y, Gopher D. Feasibility and acceptability of a new web-based cognitive training platform for cognitively healthy older adults: the breakfast task. Pilot Feasibility Stud. 2023;9(1):136. http://doi.org/10.1186/s40814-023-01359-2. PMid:37542331.

12 Quiles C, Prouteau A, Verdoux H. Assessing metacognition during or after basic-level and high-level cognitive tasks? A comparative study in a non-clinical sample. Encephale. 2020;46(1):3-6. http://doi.org/10.1016/j.encep.2019.05.007. PMid:31227210.

13 Woelfer SW, Tomitch LM, Procailo L. Working memory, metacognition and foreign language reading comprehension: a bibliographical review. Let Hoje. 2019;54(2):263-80. http://doi.org/10.15448/1984-7726.2019.2.32314.

14 Filippi R, Ceccolini A, Periche-Tomas E, Bright P. Developmental trajectories of metacognitive processing and executive function from childhood to older age. Q J Exp Psychol. 2020;73(11):1757-73. http://doi.org/10.1177/1747021820931096. PMid:32419614.

15 Gilbert SJ, Bird A, Carpenter JM, Fleming SM, Sachdeva C, Tsai PC. Optimal use of reminders: Metacognition, effort, and cognitive offloading. J Exp Psychol Gen. 2020;149(3):501-17. http://doi.org/10.1037/xge0000652. PMid:31448938.

16 Conte N, Fairfield B, Padulo C, Pelegrina S. Metacognition in working memory: confidence judgments during an N-back task. Conscious Cogn. 2023;111:103522. http://doi.org/10.1016/j.concog.2023.103522. PMid:37087901.

17 Fulton EK. How well do you think you summarize? Metacomprehension Accuracy in younger and older adults. J Gerontol B Psychol Sci Soc Sci. 2021;76(4):732-40. http://doi.org/10.1093/geronb/gbz142. PMid:31677351.

18 Arora C, Frantz C, Toglia J. Awareness of performance on a functional cognitive performance-based assessment across the adult lifespan. Front Psychol. 2021;12:753016. http://doi.org/10.3389/fpsyg.2021.753016. PMid:34803834.

19 Jacob NL, Bajaj G, Rooha A, George VM, Bhat JS. Activity ordering task: conceptualization and development of a novel context-based working memory task with a metacognitive facet. CoDAS. 2024;36(6):e20240041. http://doi.org/10.1590/2317-1782/20242024041en.

20 Sopian S, Inderawati R, Petrus I. Developing e-learning based local-folklores for eighth graders. English Rev. 2019;8(1):101-10. http://doi.org/10.25134/erjee.v8i1.1813.

21 Miyake A, Friedman NP. The nature and organization of individual differences in executive functions: four general conclusions. Curr Dir Psychol Sci. 2012;21(1):8-14. http://doi.org/10.1177/0963721411429458. PMid:22773897.

22 Horan B, Heckenberg R, Maruff P, Wright B. Development of a new virtual reality test of cognition: assessing the test-retest reliability, convergent and ecological validity of CONVIRT. BMC Psychol. 2020;8(1):61. http://doi.org/10.1186/s40359-020-00429-x. PMid:32532362.

23 Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-63. http://doi.org/10.1016/j.jcm.2016.02.012. PMid:27330520.

24 Steinborn MB, Langner R, Flehmig HC, Huestegge L. Methodology of performance scoring in the d2 sustained-attention test: cumulative-reliability functions and practical guidelines. Psychol Assess. 2018;30(3):339-57. http://doi.org/10.1037/pas0000482. PMid:28406669.

25 Oliveira IR, Seixas C, Osório FL, Crippa JAS, Abreu JN, Menezes IG, et al. Evaluation of the psychometric properties of the cognitive distortions questionnaire (CD-Quest) in a sample of undergraduate students. Innov Clin Neurosci. 2015;12(7-8):20-7. PMid:26351620.

26 Folstein MF, Folstein SE, McHugh PR. Mini-mental state: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189-98. http://doi.org/10.1016/0022-3956(75)90026-6. PMid:1202204.

27 Radhakrishnan M, Nagaraja SB. Modified Kuppuswamy socioeconomic scale 2023: stratification and updates. Int J Community Med Public Health. 2023;10(11):4415-8. http://doi.org/10.18203/2394-6040.ijcmph20233487.

28 Marian V, Blumenfeld HK, Kaushanskaya M. The language experience and proficiency questionnaire (LEAP-Q): assessing language profiles in bilinguals and multilinguals. J Speech Lang Hear Res. 2007;50(4):940-67. http://doi.org/10.1044/1092-4388(2007/067). PMid:17675598.

29 Mielicki MK, Koppel RH, Valencia G, Wiley J. Measuring working memory capacity with the letter–number sequencing task: advantages of visual administration. Appl Cogn Psychol. 2018;32(6):745-53. http://doi.org/10.1002/acp.3468.

30 Naumann S, Byrne ML, de la Fuente A, Harrewijn A, Nugiel T, Rosen M, et al. Assessing the degree of ecological validity of your study: Introducing the Multidimensional Assessment of Research in Context (MARC). Mind Brain Educ. 2022;16(3):188-98. http://doi.org/10.1111/mbe.12318.

31 Evans JD. Straightforward statistics for the behavioral sciences. Pacific Grove: Thomson Brooks/Cole Publishing Company; 1996. Linear correlation; p. 127-58.

32 Linacre JM. A user’s guide to WINSTEPS: Rasch-model computer program. Chicago, IL: MESA; 2007.

33 Schuster RM, Mermelstein RJ, Hedeker D. Acceptability and feasibility of a visual working memory task in an ecological momentary assessment paradigm. Psychol Assess. 2015;27(4):1463-70. http://doi.org/10.1037/pas0000138. PMid:25894710.

34 Devos H, Gustafson K, Ahmadnezhad P, Liao K, Mahnken JD, Brooks WM, et al. Psychometric properties of NASA-TLX and index of cognitive activity as measures of cognitive workload in older adults. Brain Sci. 2020;10(12):994. http://doi.org/10.3390/brainsci10120994. PMid:33339224.

35 Siegel ALM, Castel AD. Age-related differences in metacognition for memory capacity and selectivity. Memory. 2019;27(9):1236-49. http://doi.org/10.1080/09658211.2019.1645859. PMid:31339451.

36 D’Souza DF, Bajaj G, George VM, Karuppali S, Bhat JS. “I think I can remember” age-related changes in self-efficacy for short-term memory. J Nat Sci Biol Med. 2021;12(1):97. http://doi.org/10.4103/jnsbm.JNSBM_32_20.

37 Mitchell DJ, Cusack R. Visual short-term memory through the lifespan: preserved benefits of context and metacognition. Psychol Aging. 2018;33(5):841-54. http://doi.org/10.1037/pag0000265. PMid:30091631.

38 Fleming SM, Massoni S, Gajdos T, Vergnaud JC. Metacognition about the past and future: quantifying common and distinct influences on prospective and retrospective judgments of self-performance. Neurosci Conscious. 2016;2016(1):niw018. http://doi.org/10.1093/nc/niw018. PMid:30356936.

39 Saylik R, Raman E, Szameitat AJ. Sex differences in emotion recognition and working memory tasks. Front Psychol. 2018;9:1072. http://doi.org/10.3389/fpsyg.2018.01072. PMid:30008688.

40 Rogala J, Dreszer J, Sińczuk M, Miciuk Ł, Piątkowska-Janko E, Bogorodzki P, et al. Local variation in brain temperature explains gender-specificity of working memory performance. Front Hum Neurosci. 2024;18:1398034. http://doi.org/10.3389/fnhum.2024.1398034. PMid:39132677.

41 Bazan B. The construction and validation of a new listening span task. CEFR J Res Pract. 2021;25(1):39-56. http://doi.org/10.37546/JALTSIG.TEVAL25.1-4.

42 Krabbe PFM. The measurement of health and health status. San Diego: Academic Press; 2017. Chapter 7, Validity; p. 113-34. http://doi.org/10.1016/B978-0-12-801504-9.00007-6.
 


Submetido em:
23/07/2024

Aceito em:
23/10/2024

680ce90fa9539511644b8795 codas Articles

CoDAS

Share this page
Page Sections