COPELABS - Artigos de Revistas Internacionais com Arbitragem Científica

URI permanente para esta coleção:

http://hdl.handle.net/10437/10958

Navegar

Percorrer COPELABS - Artigos de Revistas Internacionais com Arbitragem Científica por assunto "COMPUTER SCIENCE"

A mostrar 1 - 3 de 3
  • Item
    Drop Project : an automatic assessment tool for programming assignments
    (Elsevier B.V., 2022-06-01) Cipriano, Bruno Pereira; Fachada, Nuno; Alves, Pedro; COPELABS - Cognitive and People-centric Computing
    Automated assessment tools (AATs) are software systems used in teaching environments to automate the evaluation of computer programs implemented by students. These tools can be used to stimulate the interest of computer science students in programming courses by providing quick feedback on their work and highlighting their mistakes. Despite the abundance of such tools, most of them are developed for a specific course and are not production-ready. Others lack advanced features that are required for certain pedagogical goals (e.g. Git integration) and/or are not flexible enough to be used with students having different computer literacy levels, such as first year and second year students. In this paper we present Drop Project (DP), an automated assessment tool built on top of the Maven build automation software. We have been using DP in our teaching activity since 2018, having received more than fifty thousand submissions between projects, classroom exercises, tests and homework assignments. The tool’s automated feedback has allowed us to raise the difficulty level of the course’s projects, while the grading process has become more efficient and consistent between different teachers. DP is an extensively tested, production-ready tool. The software’s code and documentation are available in GitHub under an open-source software license. Keywords: Automated assessment ;Computer science education; Programming education ; Unit testing
  • Item
    Generating multidimensional clusters with support lines
    (Elsevier B.V., 2023) Fachada, Nuno; de Andrade, Diogo; COPELABS - Cognitive and People-centric Computing
    Synthetic data is essential for assessing clustering techniques, complementing and extending real data, and allowing for more complete coverage of a given problem’s space. In turn, synthetic data generators have the potential of creating vast amounts of data – a crucial activity when real-world data is at premium – while providing a well-understood generation procedure and an interpretable instrument for methodically investigating cluster analysis algorithms. Here, we present Clugen, a modular procedure for synthetic data generation, capable of creating multidimensional clusters supported by line segments using arbitrary distributions. Clugen is open source, comprehensively unit tested and documented, and is available for the Python, R, Julia, and MATLAB/Octave ecosystems. We demonstrate that our proposal can produce rich and varied results in various dimensions, is fit for use in the assessment of clustering algorithms, and has the potential to be a widely used framework in diverse clustering-related research tasks. Keywords: Synthetic data, Clustering, Data generation, Multidimensional data
  • Item
    GPT-4.1 Sets the Standard in Automated Experiment Design Using Novel Python Libraries
    (Multidisciplinary Digital Publishing Institute (MDPI), 2025) Fachada, Nuno; Fernandes, Daniel; Fernandes, Carlos M.; Ferreira-Saraiva, Bruno D.; Matos-Carvalho, João P.; COPELABS - Cognitive and People-centric Computing; CICANT - Centre for Research in Applied Communication, Culture, and New Technologies; ECATI - School of Communication, Architecture, Arts and Information Technologies
    Large Language Models (LLMs) have advanced rapidly as tools for automating code generation in scientific research, yet their ability to interpret and use unfamiliar Python APIs for complex computational experiments remains poorly characterized. This study systematically benchmarks a selection of state-of-the-art LLMs in generating functional Python code for two increasingly challenging scenarios: conversational data analysis with the \textit{ParShift} library, and synthetic data generation and clustering using \textit{pyclugen} and \textit{scikit-learn}. Both experiments use structured, zero-shot prompts specifying detailed requirements but omitting in-context examples. Model outputs are evaluated quantitatively for functional correctness and prompt compliance over multiple runs, and qualitatively by analyzing the errors produced when code execution fails. Results show that only a small subset of models consistently generate correct, executable code, with GPT-4.1 standing out as the only model to always succeed in both tasks. In addition to benchmarking LLM performance, this approach helps identify shortcomings in third-party libraries, such as unclear documentation or obscure implementation bugs. Overall, these findings highlight current limitations of LLMs for end-to-end scientific automation and emphasize the need for careful prompt design, comprehensive library documentation, and continued advances in language model capabilities.