From 8bfe79bb9d21786edf2137e6cc1c5c2703c71056 Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Fri, 14 Jul 2023 17:00:34 +0200 Subject: [PATCH 1/7] Switches misconceptions to compile using makefiles --- research/misconceptions/Makefile | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/research/misconceptions/Makefile b/research/misconceptions/Makefile index 5c6216a2..646cc45c 100644 --- a/research/misconceptions/Makefile +++ b/research/misconceptions/Makefile @@ -1,11 +1,18 @@ +LATEXFLAGS= -shell-escape +TEX_PYTHONTEX= yes + .PHONY: all all: article.pdf article.pdf: article.tex - latexmk -shell-escape -pdf $< - article.pdf: bibliography.bib article.pdf: preamble.tex + +article.pdf: introduction.tex +article.pdf: background.tex +article.pdf: method.tex +article.pdf: results-overview.tex + article.pdf: classes.tex article.pdf: conditionals.tex article.pdf: functions-variables.tex @@ -13,11 +20,11 @@ article.pdf: problem-solving.tex article.pdf: repetitions.tex article.pdf: tools.tex article.pdf: types.tex -article.pdf: background.tex -article.pdf: introduction.tex -article.pdf: method.tex .PHONY: clean clean: latexmk -C ${RM} article.bbl article.run.xml + +INCLUDE_MAKEFILES?=../../makefiles +include ${INCLUDE_MAKEFILES}/tex.mk From a49aafd1efe07c30aa4729f29fed4422093d2411 Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Fri, 14 Jul 2023 17:01:23 +0200 Subject: [PATCH 2/7] Adds pythontex --- research/misconceptions/preamble.tex | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/research/misconceptions/preamble.tex b/research/misconceptions/preamble.tex index 82192b2d..bf943134 100644 --- a/research/misconceptions/preamble.tex +++ b/research/misconceptions/preamble.tex @@ -12,8 +12,13 @@ \usepackage[inline]{enumitem} \setlist[enumerate]{label=(\arabic*)} -\usepackage{minted} +\usepackage[outputdir=ltxobj]{minted} \setminted{autogobble,linenos} +\usepackage{pythontex} +\setpythontexfv{numbers=left} +\setpythontexoutputdir{.} +\setpythontexworkingdir{..} + \usepackage{verbatim} % adds environment for commenting out blocks of text & for better verbatim \usepackage{caption} % make it possible to include more than one captioned figure/table in a single float From 82ba470f34cec878b79a47d68edb8914b119072c Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Fri, 14 Jul 2023 17:01:57 +0200 Subject: [PATCH 3/7] Breaks results-overview out from article to own file --- research/misconceptions/article.tex | 16 +- research/misconceptions/results-overview.tex | 241 +++++++++++++++++++ 2 files changed, 242 insertions(+), 15 deletions(-) create mode 100644 research/misconceptions/results-overview.tex diff --git a/research/misconceptions/article.tex b/research/misconceptions/article.tex index a35cf0c0..15f1fb6a 100644 --- a/research/misconceptions/article.tex +++ b/research/misconceptions/article.tex @@ -32,21 +32,7 @@ \input{method} - -\section{Misconceptions in introductory programming} -\label{misconceptions} - -This section is divided to reflect the different concepts that are taught -throughout CS1. Each section will describe what students often are meant to -learn and understand in that module. Something which is followed by a -summary -of what different studies have found is difficult for students in that -particular -module and which common misconceptions that students may have. Each common -misconception will be analysed through the lens of variation theory, with -the purpose of explaining how it can be avoided by adjusting the way -the specific term or concept is being introduced or taught. - +\input{results-overview.tex} \input{functions-variables.tex} \input{classes.tex} diff --git a/research/misconceptions/results-overview.tex b/research/misconceptions/results-overview.tex new file mode 100644 index 00000000..1fda694f --- /dev/null +++ b/research/misconceptions/results-overview.tex @@ -0,0 +1,241 @@ +\section{Misconceptions in introductory programming} +\label{misconceptions} + +This section is divided to reflect the different concepts that are taught +throughout CS1. Each section will describe what students often are meant to +learn and understand in that module. Something which is followed by a +summary +of what different studies have found is difficult for students in that +particular +module and which common misconceptions that students may have. Each common +misconception will be analysed through the lens of variation theory, with +the purpose of explaining how it can be avoided by adjusting the way +the specific term or concept is being introduced or taught. + +Let us start with an example to illustrate the outline. +We will use a particular feature related to default arguments in Python that +few expect, hence most readers will hopefully not know about this and get the +intended experience. +(So this first set of patterns is not intended for students learning to program +for the first time, but rather the instructors for whom this paper is +intended.) + +Variation theory dictates that we teach using specific patterns of variation. +We should start with a contrast pattern in a critical aspect, followed by a +generalization pattern for the same critical aspect and finally tie several +aspects together using a fusion pattern. + +Let's get started. + +\begin{description} + \item[Contrast] The contrast pattern requires two examples to create + contrast. + The left-hand example should be read first and then the right-hand + example will highlight the changes made to the left-hand example to get + the right-hand one. + (So that one can make the changes oneself.) + + We assume that the reader is familiar with default values for arguments in + Python. + We define a function \mintinline{python}{expand} that takes a list as an + argument and expands it by appending an element \mintinline{python}{1} + (left-hand code below). + Then we run some examples. + + \hfill + \begin{minipage}[t]{0.45\columnwidth} + \begin{pyblock}[default1] +def expand(x=[]): + return x + [1] + + +print( + f"[1] -> {expand([1])}\n" + f"[2] -> {expand([2])}\n" + f"() -> {expand()}\n" + f"() -> {expand()}\n" +) + \end{pyblock} + \vspace{0.5em} + This yields the output + \vspace{0.5em} + \printpythontex[verbatim] + \end{minipage} + \hfill + \begin{minipage}[t]{0.45\columnwidth} + \begin{pyblock}[default2][highlightlines={2-3}] +def expand(x=[]): + x += [1] + return x + +print( + f"[1] -> {expand([1])}\n" + f"[2] -> {expand([2])}\n" + f"() -> {expand()}\n" + f"() -> {expand()}\n" +) + \end{pyblock} + + \vspace{0.5em} + This yields the output + \vspace{0.5em} + \printpythontex[verbatim][highlightlines={4}] + \end{minipage} + \hfill + + On the right-hand side, the change is that we modify \mintinline{python}{x} + before returning the new value. + However, we notice in the output that something weird happens when we use + the default value of the parameter now: + it seems like we actually update the default value every time we run the + function using the default value. + + \item[Generalisation] This brings us to the generalisation pattern. + In the generalisation pattern we vary the non-critical aspects and keep the + critical aspect invariant. + In this case, we just add more print statements to the example. + (For brevity we don't repeat the definition of the + function~\mintinline{python}{expand}.) + + \hfill + \begin{minipage}[t]{0.45\columnwidth} + \begin{pyblock}[default1] +print( + f"() -> {expand()}\n" + f"[3] -> {expand([3])}\n" + f"() -> {expand()}\n" + f"[] -> {expand([])}\n" + f"[] -> {expand([])}\n" + f"() -> {expand()}\n" +) + \end{pyblock} + \vspace{0.5em} + This yields the output + \vspace{0.5em} + \printpythontex[verbatim] + \end{minipage} + \hfill + \begin{minipage}[t]{0.45\columnwidth} + \begin{pyblock}[default2] +print( + f"() -> {expand()}\n" + f"[3] -> {expand([3])}\n" + f"() -> {expand()}\n" + f"[] -> {expand([])}\n" + f"[] -> {expand([])}\n" + f"() -> {expand()}\n" +) + \end{pyblock} + + \vspace{0.5em} + This yields the output + \vspace{0.5em} + \printpythontex[verbatim][highlightlines={1,3,6}] + \end{minipage} + \hfill + + We can see in the output that the default value in the argument keeps + expanding. + In a sense, the generalisation pattern is the same as induction or the + scientific method. + + Now, we'll change the example, but keep this property invariant; + or, phrased in terms of the scientific method, we try to falsify our + hypothesis. + \begin{pyblock}[default1][highlightlines={15-16}] +class Person: + def __init__(self, first, last): + self.first = first + self.last = last + +def create_person(first, last, + person_base=Person("Gina", "Joe")): + person_base.first = first + person_base.last = last + return person_base + +person_A = create_person("A", "Ason") +print(person_A.first) +person_B = create_person("B", "Bdotter") +print(person_B.first) +print(person_A.first) + \end{pyblock} + \vspace{0.5em} + This yields the output + \vspace{0.5em} + \printpythontex[verbatim][highlightlines={2-3}] + + We can see that the two last lines of the output is the same. + This means that they refer to the same object, \ie that + \mintinline{python}{person_A} and \mintinline{python}{person_B} refer to + the same object. + + \item[Fusion] Now we can fuse this back with our previous understanding of + default arguments. + \begin{pyblock}[default1] +def increment(x=1): + x += 1 + return x + +print(f"(1) -> {increment(1)}") +print(f"(2) -> {increment(2)}") +print(f"() -> {increment()}") +print(f"() -> {increment()}") +print(f"() -> {increment()}") + \end{pyblock} + \vspace{0.5em} + This yields the output + \vspace{0.5em} + \printpythontex[verbatim][highlightlines={3-5}] + + And we can thus conclude that this phenomenon happens only with mutable + objects. + The reason is how Python is constructed. + The default value is constructed when the function is defined and then + referenced whenever the function is called without the argument. + Consider this final example. + + \begin{pyblock}[default1][highlightlines=3] +class TraceClass: + def __init__(self): + print(f"{self} created") + +def test_function(obj=TraceClass()): + print(f"test_function called with obj = {obj}") + +print("Test code begins") +test_function() + \end{pyblock} + \vspace{0.5em} + This yields the output + \vspace{0.5em} + \printpythontex[verbatim][highlightlines={1}] + + We see that the print statement from the constructor is executed before the + test code is executed, supporting our thesis that the default value is + constructed when the function is defined, not the when function is called, + and then referenced throughout. +\end{description} + +There are several things to note with this example. +First, note how the contrast pattern is designed to focus the your (the +reader's) attention to the phenomenon at hand: that default values can change +during execution. +Next, the generalisation pattern broadens our view of when this phenomenon +happens, that the objects are referenced and reused. +Finally, the fusion pattern merges this back into our original view of default +values, namely that they work as usual for non-mutable types (\eg integers). + +Second thing to note, if you are a seasoned programmer, once you had that +initial contrast pattern the remaining patterns (generalisation and fusion) +probably resembles quite a lot what you would have tested yourself to make +sense of this phenomenon. +It would probably resemble how you would go about to \enquote{debug} this. + +XXX do a quick test on colleagues how they would debug this example. + +XXX add reference to NCOL that students taught according to variation theory, +becomes better at creating these patterns for themselves. +In the context of programming, this should mean that if we teach them using +variation theory, they would get better at debugging. + From be6f30803917881e110990922673e7ac848962b4 Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Sat, 15 Jul 2023 19:39:43 +0200 Subject: [PATCH 4/7] Improves initial example, rearranges it --- research/misconceptions/results-overview.tex | 88 +++++++++++--------- research/misconceptions/test.cpp | 20 +++++ 2 files changed, 69 insertions(+), 39 deletions(-) create mode 100644 research/misconceptions/test.cpp diff --git a/research/misconceptions/results-overview.tex b/research/misconceptions/results-overview.tex index 1fda694f..79ae4d6f 100644 --- a/research/misconceptions/results-overview.tex +++ b/research/misconceptions/results-overview.tex @@ -142,58 +142,46 @@ \section{Misconceptions in introductory programming} Now, we'll change the example, but keep this property invariant; or, phrased in terms of the scientific method, we try to falsify our hypothesis. - \begin{pyblock}[default1][highlightlines={15-16}] + \begin{pyblock}[default1][highlightlines={10,21-22}] class Person: def __init__(self, first, last): self.first = first self.last = last -def create_person(first, last, - person_base=Person("Gina", "Joe")): - person_base.first = first - person_base.last = last + def __str__(self): + return f"{self.first} {self.last}" + +def create_person(first=None, last=None, + person_base=Person("Gina", "Jones")): + if first: person_base.first = first + if last: person_base.last = last return person_base -person_A = create_person("A", "Ason") -print(person_A.first) -person_B = create_person("B", "Bdotter") -print(person_B.first) -print(person_A.first) +person_default = create_person() +print(person_default) +person_A = create_person("Ada") +print(person_A) +person_B = create_person("Beda") +print(person_B) +print(person_A) +print(person_default) \end{pyblock} \vspace{0.5em} This yields the output \vspace{0.5em} - \printpythontex[verbatim][highlightlines={2-3}] + \printpythontex[verbatim][highlightlines={4-5}] - We can see that the two last lines of the output is the same. + We can see that the last three lines of the output is the same. This means that they refer to the same object, \ie that - \mintinline{python}{person_A} and \mintinline{python}{person_B} refer to - the same object. - - \item[Fusion] Now we can fuse this back with our previous understanding of - default arguments. - \begin{pyblock}[default1] -def increment(x=1): - x += 1 - return x - -print(f"(1) -> {increment(1)}") -print(f"(2) -> {increment(2)}") -print(f"() -> {increment()}") -print(f"() -> {increment()}") -print(f"() -> {increment()}") - \end{pyblock} - \vspace{0.5em} - This yields the output - \vspace{0.5em} - \printpythontex[verbatim][highlightlines={3-5}] + \mintinline{python}{person_A}, \mintinline{python}{person_B} and + \mintinline{python}{person_default} all refer to the same object. - And we can thus conclude that this phenomenon happens only with mutable - objects. - The reason is how Python is constructed. - The default value is constructed when the function is defined and then - referenced whenever the function is called without the argument. - Consider this final example. + This must be due to how Python is constructed. + The default value seems to be constructed when the function is defined, not + when it's called (like in other languages, C++ for instance), and then + referenced (not copied) whenever the function is called without the + argument. + Consider this example. \begin{pyblock}[default1][highlightlines=3] class TraceClass: @@ -212,9 +200,31 @@ \section{Misconceptions in introductory programming} \printpythontex[verbatim][highlightlines={1}] We see that the print statement from the constructor is executed before the - test code is executed, supporting our thesis that the default value is + test code is executed, supporting our hypothesis that the default value is constructed when the function is defined, not the when function is called, and then referenced throughout. + + \item[Fusion] Now we can fuse this back with our previous understanding of + default arguments, to see that it doesn't work with non-mutables like + integers. + \begin{pyblock}[default1] +def increment(x=1): + x += 1 + return x + +print(f"(1) -> {increment(1)}") +print(f"(2) -> {increment(2)}") +print(f"() -> {increment()}") +print(f"() -> {increment()}") +print(f"() -> {increment()}") + \end{pyblock} + \vspace{0.5em} + This yields the output + \vspace{0.5em} + \printpythontex[verbatim][highlightlines={3-5}] + + And we can thus conclude that this phenomenon happens only with mutable + objects. \end{description} There are several things to note with this example. diff --git a/research/misconceptions/test.cpp b/research/misconceptions/test.cpp new file mode 100644 index 00000000..5a6e6adf --- /dev/null +++ b/research/misconceptions/test.cpp @@ -0,0 +1,20 @@ +#include + +class TraceClass { + public: + TraceClass() { + std::cout << this << " object created" << std::endl; + } +}; + +void test_function(TraceClass obj=TraceClass()) { + std::cout << "test_function called" << std::endl; +} + +int main(void) { + std::cout << "test code starts" << std::endl; + test_function(); + test_function(); + + return 0; +} From a5e0ebcfcc431d5b63a813ee9cfa7d8e4a00506f Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Sat, 15 Jul 2023 19:49:02 +0200 Subject: [PATCH 5/7] Further improvements to initial example --- research/misconceptions/results-overview.tex | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/research/misconceptions/results-overview.tex b/research/misconceptions/results-overview.tex index 79ae4d6f..4ec49a1e 100644 --- a/research/misconceptions/results-overview.tex +++ b/research/misconceptions/results-overview.tex @@ -142,7 +142,9 @@ \section{Misconceptions in introductory programming} Now, we'll change the example, but keep this property invariant; or, phrased in terms of the scientific method, we try to falsify our hypothesis. - \begin{pyblock}[default1][highlightlines={10,21-22}] + (We highlight the lines that keeps this property, \ie remains invariant in + the variation theoretic sense.) + \begin{pyblock}[default1][highlightlines={10-12}] class Person: def __init__(self, first, last): self.first = first @@ -172,9 +174,10 @@ \section{Misconceptions in introductory programming} \printpythontex[verbatim][highlightlines={4-5}] We can see that the last three lines of the output is the same. - This means that they refer to the same object, \ie that - \mintinline{python}{person_A}, \mintinline{python}{person_B} and - \mintinline{python}{person_default} all refer to the same object. + The first is expected, but the last two (highlighted) indicates that they + refer to the same object; \ie that \mintinline{python}{person_A}, + \mintinline{python}{person_B} and \mintinline{python}{person_default} all + refer to the same object. This must be due to how Python is constructed. The default value seems to be constructed when the function is defined, not @@ -183,7 +186,7 @@ \section{Misconceptions in introductory programming} argument. Consider this example. - \begin{pyblock}[default1][highlightlines=3] + \begin{pyblock}[default1][highlightlines=5] class TraceClass: def __init__(self): print(f"{self} created") @@ -200,9 +203,9 @@ \section{Misconceptions in introductory programming} \printpythontex[verbatim][highlightlines={1}] We see that the print statement from the constructor is executed before the - test code is executed, supporting our hypothesis that the default value is - constructed when the function is defined, not the when function is called, - and then referenced throughout. + test code is executed (highlighted line), supporting our hypothesis that + the default value is constructed when the function is defined, not the when + function is called, and then referenced throughout. \item[Fusion] Now we can fuse this back with our previous understanding of default arguments, to see that it doesn't work with non-mutables like From 9765d2e582cbdb0ea39a786fdb3c8e17e38aaa4d Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Fri, 28 Jul 2023 09:15:33 +0200 Subject: [PATCH 6/7] Removes unnecessary `\hfill`s --- research/misconceptions/results-overview.tex | 4 ---- 1 file changed, 4 deletions(-) diff --git a/research/misconceptions/results-overview.tex b/research/misconceptions/results-overview.tex index 4ec49a1e..f8f330ae 100644 --- a/research/misconceptions/results-overview.tex +++ b/research/misconceptions/results-overview.tex @@ -42,7 +42,6 @@ \section{Misconceptions in introductory programming} (left-hand code below). Then we run some examples. - \hfill \begin{minipage}[t]{0.45\columnwidth} \begin{pyblock}[default1] def expand(x=[]): @@ -81,7 +80,6 @@ \section{Misconceptions in introductory programming} \vspace{0.5em} \printpythontex[verbatim][highlightlines={4}] \end{minipage} - \hfill On the right-hand side, the change is that we modify \mintinline{python}{x} before returning the new value. @@ -97,7 +95,6 @@ \section{Misconceptions in introductory programming} (For brevity we don't repeat the definition of the function~\mintinline{python}{expand}.) - \hfill \begin{minipage}[t]{0.45\columnwidth} \begin{pyblock}[default1] print( @@ -132,7 +129,6 @@ \section{Misconceptions in introductory programming} \vspace{0.5em} \printpythontex[verbatim][highlightlines={1,3,6}] \end{minipage} - \hfill We can see in the output that the default value in the argument keeps expanding. From c9f490d3f8336f66f7644196e0b01a95ced79ced Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Fri, 28 Jul 2023 09:22:41 +0200 Subject: [PATCH 7/7] Adds results from first debug test --- research/misconceptions/results-overview.tex | 24 ++++++++++++++++---- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/research/misconceptions/results-overview.tex b/research/misconceptions/results-overview.tex index f8f330ae..5e160164 100644 --- a/research/misconceptions/results-overview.tex +++ b/research/misconceptions/results-overview.tex @@ -64,7 +64,7 @@ \section{Misconceptions in introductory programming} \begin{minipage}[t]{0.45\columnwidth} \begin{pyblock}[default2][highlightlines={2-3}] def expand(x=[]): - x += [1] + x.extend([1]) return x print( @@ -238,10 +238,24 @@ \section{Misconceptions in introductory programming} Second thing to note, if you are a seasoned programmer, once you had that initial contrast pattern the remaining patterns (generalisation and fusion) probably resembles quite a lot what you would have tested yourself to make -sense of this phenomenon. -It would probably resemble how you would go about to \enquote{debug} this. - -XXX do a quick test on colleagues how they would debug this example. +sense of this phenomenon---it would probably resemble how you would go about to +\enquote{debug} this. + +We actually tested this hypothesis on several colleagues who have been +programming for many years, are well-versed in Python but didn't know of this +phenomenon. +We gave them the contrast above and asked them to \enquote{debug} this +behaviour and later explain it when they understood it. +(To record the data, we asked them to think aloud and recorded their screen and +voice in a Zoom session.) + +% XXX Test with more colleagues. +% Tested with Alexander. +The concrete examples that they tried varied from person to person, some tried +many more examples than above, but the patterns of variation shown above were +present. +Indeed, they couldn't explain the phenomenon until they had generated all the +patterns above---covering both generalisation and fusion. XXX add reference to NCOL that students taught according to variation theory, becomes better at creating these patterns for themselves.