Publications

Publications

If  you cannot see the papers (the wheels are spinning), please visit http://eduinf.eu/publications/. Last update 16 June 2020.

2020

Alrashidi, H., Joy, M., Ullmann, T. D., & Almujally, N. (2020). Validating the Reflective Writing Framework (RWF) for Assessing Reflective Writing in Computer Science Education Through Manual Annotation. In V. Kumar & C. Troussas (Eds.), Intelligent Tutoring Systems (pp. 323–326). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-49663-0_38
Alrashidi, H., Joy, M., Ullmann, T. D., & Almujally, N. (2020). Educators’ Validation on a Reflective Writing Framework (RWF) for Assessing Reflective Writing in Computer Science Education. In V. Kumar & C. Troussas (Eds.), Intelligent Tutoring Systems (pp. 316–322). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-49663-0_37
Alrashidi, H., Ullmann, T. D., Ghounaim, S., & Joy, M. (2020). A Framework for Assessing Reflective Writing Produced Within the Context of Computer Science Education. In Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20. Frankfurt, Germany.
Ullmann, T. D. (2020). Propensity score analysis of the L161 peer assessment activity (QEI report). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from https://openuniv.sharepoint.com/sites/units/lds/scholarship-exchange/documents/IET_QEI_Report_20_2_Peer_assessment_v1.pdf
Kubiak, C., & Ullmann, T. D. (2020). Student support advisors: context, practice and impact (QEI report). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from https://openuniv.sharepoint.com/sites/units/lds/scholarship-exchange/documents/IET_QEI_Report_20_1_Student_support_advisors_v1.pdf

2019

Ullmann, T. D. (2019). Automated Analysis of Reflection in Writing: Validating Machine Learning Approaches. International Journal of Artificial Intelligence in Education, 1–41. https://doi.org/10.1007/s40593-019-00174-2
Ullmann, T. D., Liddo, A. D., & Bachler, M. (2019). A Visualisation Dashboard for Contested Collective Intelligence Learning Analytics to Improve Sensemaking of Group Discussion. RIED. Revista Iberoamericana de Educación a Distancia, 22(1). https://doi.org/10.5944/ried.22.1.22294

2018

Ullmann, T. D., Lay, S., Cross, S., Gaved, M., Jones, E., Hidalgo, R., … Rienties, B. (2018). Scholarly insight Spring 2018: a Data wrangler perspective (Scholarly Insight Series). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://oro.open.ac.uk/56732/
Ullmann, T. D., Lay, S., Coughlan, T., Lister, K., Cross, S., Rienties, B., & Whitelock, D. (2018). Understanding SEaM student comments from a Big(ger) Data perspective: what are students saying? (p. 39). Presented at the CALRG Annual Conference 2018, The Open University, Milton Keynes: Computers and Learning Research Group. Retrieved from http://www.open.ac.uk/blogs/CALRG/wp-content/uploads/2018/06/CALRG-2018-Conference-proceedings_v3.pdf
Charitonos, K., Collier, C., Edwards, C., Kukulska-Hulme, A., Marr, L., Mayles, K., … Ullmann, T. D. (2018). Group Tuition Policy - Academic Evaluation (Report of Education Committee Working Group on the Academic Evaluation of the Group Tuition Policy (GTP) to the Senate No. S-2018-01-10). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://css2.open.ac.uk/ecms/get.aspx?object_id=090173b48161457c&format=pdf

2017

Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2017). DW - Data wrangler’s key metric report for WELS (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2017). DW - Data wrangler’s key metric report for STEM (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2017). DW - Data wrangler’s key metric report for FBL (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2017). DW - Data wrangler’s key metric report for FASS (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Ullmann, T. D. (2017). Group tuition policy - Analysis of SEaM open-ended questions (Report to the Group Tuition Policy evaluation working group). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from https://intranet9.open.ac.uk/collaboration/Scholarship-Exchange/Wiki/Document.aspx?DocumentID=2174
Ullmann, T. D., Marsh, V., Cross, S., & Rienties, B. (2017). DW - Data wrangler’s Nations report for Spring. Milton Keynes: The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Nation%20Report%20Spring%202017/DataWranglerNationReport_Spring_2017.html
Ullmann, T. D., Lay, S., & Rienties, B. (2017). DW - Data wrangler’s Nations report for Autumn. Milton Keynes: The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Nations%20Report%20Autumn%202017/DataWranglerNationsReport_Autumn_2017.html
Rienties, B., Rogaten, J., Nguyen, Q., Edwards, C., Gaved, M., Holt, D., … Ullmann, T. (2017). Scholarly Insight Spring 2017: a Data Wrangler Perspective (Scholarly Insight Series). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://oro.open.ac.uk/49648/
Rienties, B., Clow, D., Coughlan, T., Cross, S., Edwards, C., Gaved, M., … Ullmann, T. (2017). Scholarly insight Autumn 2017: a Data wrangler perspective (Scholarly Insight Series). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://oro.open.ac.uk/52226/
Rienties, B., Cross, S., Marsh, V., & Ullmann, T. (2017). Making sense of learner and learning Big Data: reviewing 5 years of Data Wrangling at the Open University UK. Open Learning: The Journal of Open, Distance and e-Learning. Retrieved from http://oro.open.ac.uk/49085/
Ullmann, T. D., Lay, S., & Rienties, B. (2017). DW - Data wrangler’s key metric report for LTI. Milton Keynes: The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202017/
Kravcik, M., Mikroyannidis, A., Pammer, V., Prilla, M., & Ullmann, T. D. (Eds.). (2017). Special Issue on: Awareness and Reflection in Technology Enhanced Learning. International Journal of Technology Enhanced Learning, 9(3/4). Retrieved from http://www.inderscience.com/browse/getEditorial.php?articleID=5145
Coughlan, T., Ullmann, T. D., & Lister, K. (2017). Understanding Accessibility as a Process through the Analysis of Feedback from Disabled Students. In W4A’17 International Web for All Conference. New York, USA: ACM. Retrieved from http://oro.open.ac.uk/48991/
Ullmann, T. D. (2017). Reflective Writing Analytics: Empirically Determined Keywords of Written Reflection. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 163–167). New York, NY, USA: ACM. https://doi.org/10.1145/3027385.3027394

2016

Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2016). DW - Data wrangler’s key metric report for FASS (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2016). DW - Data wrangler’s key metric report for WELS (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2016). DW - Data wrangler’s key metric report for STEM (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2016). DW - Data wrangler’s key metric report for FBL (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Rienties, B., Edwards, C., Gaved, M., Marsh, V., Herodotou, C., Clow, D., … Ullmann, T. D. (2016). Scholarly insight 2016: a Data wrangler perspective (Scholarly Insight Series). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://oro.open.ac.uk/48244/
Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2016). DW - Data wrangler’s key metric report for CICP (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Ullmann, T. D., Marsh, V., Slee, A., Cross, S., & Rienties, B. (2016). DW - Data wrangler’s key metric report for IET (Key Metric reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Key%20Metrics%20Report%20Autumn%202016/
Ullmann, T. D. (2016). FELS CYE IET Wranglers Report Spring 2016. In-depth analysis of ED841 in the context of its qualifications (Data Wrangler reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Spring%202016/FELS%20Education%20-%20IET%20Wranglers%20Report%20Spring%202016_final.pdf
Brasher, A., Cooper, A., Ferguson, R., Hillaire, G., Mittelmeier, J., Rienties, B., & Ullmann, T. D. (2016). LAEP - Inventory and literature review, including a proposal for the case study templates (Deliverable No. D2.2). Milton Keynes: Institute of Educational Technology, The Open University.
Brasher, A., Ferguson, R., Hillaire, G., Mittelmeier, J., Rienties, B., & Ullmann, T. D. (2016). LAEP - In-Depth Case Studies (Deliverable No. D3). Milton Keynes: Institute of Educational Technology, The Open University.
Ferguson, R., Brasher, A., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B., & Ullmann, T. D. (2016). LAEP - The Implications and Opportunities of Learning Analytics for European Educational Policy (Deliverable No. D5.2). Milton Keynes: Institute of Educational Technology, The Open University.
Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., … Vuorikari, R. (2016). Research Evidence on the Use of Learning Analytics - Implications for Education Policy. (R. Vuorikari & J. Castano Munoz, Eds.). Publications Office of the European Union. Retrieved from https://ec.europa.eu/jrc/en/publication/eur-scientific-and-technical-research-reports/research-evidence-use-learning-analytics-implications-education-policy
Kravcik, M., Mikroyannidis, A., Pammer, V., Prilla, M., & Ullmann, T. D. (Eds.). (2016). Proceedings of the 6th Workshop on Awareness and Reflection in Technology Enhanced Learning In conjunction with the 11th European Conference on Technology Enhanced Learning: Adaptive and Adaptable Learning. Lyon, France: CEUR-WS.org. Retrieved from http://ceur-ws.org/Vol-1736/

2015

Ullmann, T. D. (2015). Keywords of written reflection - a comparison between reflective and descriptive datasets. In Proceedings of the 5th Workshop on Awareness and Reflection in Technology Enhanced Learning (Vol. 1465, pp. 83–96). Toledo, Spain: CEUR-WS.org. Retrieved from http://ceur-ws.org/Vol-1465/paper8.pdf
Ullmann, T. D. (2015). Automated detection of reflection in texts. A machine learning based approach (PhD Thesis). The Open University. Retrieved from http://oro.open.ac.uk/45402/
Ullmann, T. D. (2015). FELS CYE Studies Data Wrangling Report Autumn 2015: Individual module and programme level analysis (Data Wrangler reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Autumn%202015/FELS%20Education%20-%20%20IET%20%20Wranglers%20Report%20Autumn%202015.pdf
Ullmann, T. D. (2015). FELS Language Studies Data Wrangling Report Autumn 2015: Comparative analysis of individual modules and qualification focused analysis (Data Wrangler reports). Milton Keynes: Institute of Educational Technology, The Open University. Retrieved from http://article.iet.open.ac.uk/D/Data%20Wranglers/Autumn%202015/FELS%20Languages%20-%20IET%20Wranglers%20Report%20Autumn%202015.pdf
Parent, M.-A., Liddo, A. D., Ullmann, T. D., & Klein, M. (2015). Catalyst - Project Testbed: Argument Mapping and Deliberation Analytics (Deliverable No. D4.2b). Retrieved from http://catalyst-fp7.eu/wp-content/uploads/2015/11/CATALYST_D4.3.pdf
Kravcik, M., Mikroyannidis, A., Pammer, V., Prilla, M., & Ullmann, T. D. (Eds.). (2015). Proceedings of the 5th Workshop on Awareness and Reflection in Technology Enhanced Learning In conjunction with the 10th European Conference on Technology Enhanced Learning: Design for Teaching and Learning in a Networked World, ARTEL@ECTEL 2015, Toledo, Spain, September 15, 2015 (Vol. 1465). CEUR-WS.org. Retrieved from http://ceur-ws.org/Vol-1465

2014

Ullmann, T. D., Liddo, A. D., & Bachler, M. (2014). Catalyst - Collective Intelligence Analytics Dashboard Usability Evaluation (Deliverable No. D4.6). The Open University. Retrieved from http://catalyst-fp7.eu/wp-content/uploads/2014/12/CATALYST_-D4.6.pdf
Kravcik, M., Mikroyannidis, A., Pammer, V., Prilla, M., Ullmann, T. D., & Wild, F. (Eds.). (2014). Proceedings of the 4th workshop on awareness and reflection in technology enhanced learning. Graz, Austria: CEUR-WS.org. Retrieved from http://ceur-ws.org/Vol-1238/

2013

Ullmann, T. D., Wild, F., & Scott, P. (2013). Reflection - quantifying a rare good. In M. Kravcik, B. R. Krogstie, A. Moore, V. Pammer, L. Pannese, M. Prilla, … T. D. Ullmann (Eds.), Proceedings of the 3rd Workshop on Awareness and Reflection in Technology-Enhanced Learning (pp. 29–40). Paphos, Cyprus: CEUR-WS.org. Retrieved from http://ceur-ws.org/Vol-1103/paper2.pdf
Kravcik, M., Krogstie, B. R., Moore, A., Pammer, V., Pannese, L., Prilla, M., … Ullmann, T. D. (Eds.). (2013). Proceedings of the 3rd Workshop on Awareness and Reflection in Technology-Enhanced Learning. Paphos, Cyprus. Retrieved from http://ceur-ws.org/Vol-1103/
Meyer, P., Kelle, S., Ullmann, T. D., Scott, P., & Wild, F. (2013). Interdisciplinary Cohesion of TEL – An Account of Multiple Perspectives. In D. Hernández-Leo, T. Ley, R. Klamma, & A. Harrer (Eds.), Scaling up Learning for Sustained Impact (pp. 219–232). Springer Berlin Heidelberg. Retrieved from http://link.springer.com/chapter/10.1007/978-3-642-40814-4_18

2012

Ullmann, T. D., Wild, F., & Scott, P. (2012). Comparing Automatically Detected Reflective Texts with Human Judgements. In A. Moore, V. Pammer, L. Pannese, M. Prilla, K. Rajagopal, W. Reinhardt, … C. Voigt (Eds.), 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning. Saarbruecken, Germany: CEUR-WS.org. Retrieved from http://ceur-ws.org/Vol-931/paper8.pdf
Ullmann, T. (2012, March 15). Co-citation analysis of the topic Social Network Analysis. Retrieved November 25, 2012, from http://eduinf.eu/2012/03/15/co-citation-analysis-of-the-topic-social-network-analysis/
Aceto, S., Proli, D., Padron Napoles, C., Persico, D., Pozzi, F., Herder, E., … Ullmann, T. D. (2012). STELLAR D5.3.3 Report on Community Building with Stakeholders (Deliverable). Retrieved from http://www.stellarnet.eu/kmi/deliverables/20120803_stellar_d5.3.3_final.pdf
Vries, F., Crook, C., Aceto, S., Padron Napoles, C., Persico, D., Specht, M., … Ullmann, T. D. (2012). STELLAR D5.4 Summative Evaluation, stakeholder community and its sustainability (Deliverable). Retrieved from http://www.stellarnet.eu/index.php/repository/deliverable_repository_list/
Wild, F., Ullmann, T. D., Anastasiou, L., Scott, P., Kraker, P., Lindstaedt, S., … Balacheff, N. (2012). STELLAR D6.6 The STELLAR Research 2.0 Infrastructure (Deliverable). Retrieved from http://www.stellarnet.eu/index.php/repository/deliverable_repository_list/
Ullmann, T. D. (2012). On the automated detection of reflective writings. In Proceedings of the 2012 CRC PhD Student Conference. Milton Keynes.
Moore, A., Pammer, V., Prilla, M., Rajagopal, K., Reinhardt, W., Ullmann, T. D., & Voigt, C. (Eds.). (2012). Proceedings of the 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning. Saarbrücken, Germany. Retrieved from http://ceur-ws.org/Vol-931/
Sie, R. L. L., Ullmann, T. D., Rajagopal, K., Cela, K., Bitter-Rijpkema, M., & Sloep, P. B. (2012). Social network analysis for technology-enhanced learning: review and future directions. International Journal of Technology Enhanced Learning, 4(3), 172–190. https://doi.org/10.1504/IJTEL.2012.051582

2011

2010

2009