Autori
Zamparelli, Roberto
Chesi, Cristiano
Vespignani, Francesco

Titolo
Modelli generativi e sintassi generativa
Periodico
Sistemi intelligenti
Anno: 2023 - Fascicolo: 2 - Pagina iniziale: 329 - Pagina finale: 350

In this short paper we present the results of four experiments assessing various degree of morphosyntactic and semantic linguistic competence in three very large language models (LLMs), namely davinci (GPT-3/ChatGPT), davinci-002 and davinci-003 (GPT-3.5 with different training options). We focused on (i) acceptability, (ii) complexity and (iii) coherence judgments on 7-point Likert scales and on (iv) syntactic development by means of a forced choice task. The datasets used are taken from available test-sets presented in shared tasks by the NLP community or from linguistic tests. The results suggest that, despite a rather good performance overall, these LLMs cannot be considered competence models since they do not qualify neither as descriptively nor explanatorily adequate



SICI: 1120-9550(2023)2<329:MGESG>2.0.ZU;2-7
Testo completo: https://www.rivisteweb.it/download/article/10.1422/108133
Testo completo alternativo: https://www.rivisteweb.it/doi/10.1422/108133

Esportazione dati in Refworks (solo per utenti abilitati)

Biblioteche ACNP che possiedono il periodico