Step 5: Data Analysis

Step five of six in my research process has arrived.  This the data analysis step, or as Norris & Ortega describe it, the scoring research observations step.  But where to begin?

To focus my thinking, I have returned to my research proposal which says:

“The first drafts in the first writing units will be graded according to NCEA criteria to allow placement of students into four groups according to ability which will provide benchmarks for the research. The first drafts in both writing units will then act as pre-tests and the second drafts as post-tests. Situated error, complexity and accuracy measures, which were trialled in a 2013 pilot study, will be used to calculate language development between the two drafts of each unit. A sufficient increase in development should encourage FL teachers to make time in their programmes for students to reflect as they write.”

In practical terms this means I will:

A: Second Language Writing Data

  1. Create an XL spreadsheet with school then student names and the NCEA grade given to their first draft of Task 1 (benchmark).  XL will allow sorting according to school and benchmark as well as intervention and non-intervention groups.
  2. Take all the Task 1 intervention first and second drafts and measure change between them using situated error, then complexity and then accuracy measures.
  3. Take all the Task 1 non-intervention first and second drafts and measure change between them using situated error, then complexity and then accuracy measures.
  4. Take all the Task 2 intervention first and second drafts and measure change between them using situated error, then complexity and then accuracy measures.
  5. Take all the Task 2 non-intervention first and second drafts and measure change between them using situated error, then complexity and then accuracy measures.
  6. Measure the difference between the intervention and the non-intervention participant data for Task 1 in the situated error, then complexity and then accuracy measures.
  7. Measure the difference between the intervention and the non-intervention participant data for Task 2 in the situated error, then complexity and then accuracy measures.
  8. Conclude if there is any significant difference between the intervention and non-intervention groups in both Tasks 1 and 2.
  9. Use Aljaafreh and Lantolf’s Regulatory Scale to measure the depth of the individual students’ responses to the feedback given by the teacher on the first draft.

B: First Language Reflection Data

  1. Prepare students’ digital reflections for analysis by removing the guiding questions and numbering (so only student words remain)
  2. Analyse the students’ reflections with Wordle (or another word frequency tool) to find key words and themes.
  3. If useful, use other quantifying tools to analyse the reflections e.g. co-occurrences (concordancing), distribution of words and/or collocations
  4. Analyse the reflections for depth of thought using Biggs & Collis’ SOLO Taxonomy
  5. Analyse the reflections for level of engagement using Cohen & Dörnyei’s Motivational Instrument
  6. Analyse the reflections using Kember et al’s Levels of Reflection Scheme
  7. Consider Metaphor Analysis

C: Combined Writing and Reflection

  1. Compare progress between the first and second drafts of individual students (case studies) with the quality of their reflections.

D: Return to my “Tools for Locating the Learning,” quadrant and evaluate the tools used in the research.

Quadrant of Tools for Locating the Learning

After my two terms of data collection in four secondary schools, I was concerned I might not have enough data to work with, but I think, now, after breaking things down as above, I have enough to keep me going for some time.

I just have to get started!!!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s