Up now: David Gibson at #P3Workshop on Assessment and Digital Media Learning

(David Gibson was one of the recipients of a HASTAC/MacArthur Foundation Digital Media & Learning Competition award for Global Challenge Award, so it's nice to see him here presenting his academic work.)

Assessment & Digital Media Learning

New possibilities for assessment stem from evidence entered design theory, traditional decisions and dilemmas of assessment, and the affordances of immersive digital media learning environment.

Behaviorism, Cognitivism, Constructivism --> we're going to look more closely within Cognitivism at: Learner, Knowledge, Community and Assessment. How are learners as individuals put together? Community is something tha twe think of as some intersecting, overlapping entities. The knowledge community with special language, concepts and practices -- they are the ones who should be the ones credentialing. Knowledge: epistemological sources: explantory, deductive, conceptual, analogical, perceptual. David is interested in this because he wants to understand how this could work in a digital media environment. Most of our current assessment is in explanatory and deductive, which leaves out a whole lot of knowledge, such as conceptual, analogical and perceptual. Assessment: qualitative-complex systems -- quantiatitve...tests, observations, examinations.

Assessment, community, learner, knowledge: project and challenge-based learning. Self-directed personalized learning, peer to peer pedagogy, and ePortfolios. It would be interesting to see a curriculum that was developed to support these four ideas.

• Digital Media Assessment Theory: David proposes that there might be a theory like this one day. The following might be included:
• Network-based.
• Online data gathering
• Distributed interoperating web services.
• Assessment.  Measures and feedback for improvement of performance and evaluating learners, that are multidimensional, integrated, and revealed in performance over time." Walvoord and Anderson, 1998.

Primary proposition:

1. The mediational power and invisibility of technology can be used to create new forms of instruction and assessment.

2. The new forms allow unobtrusive assessment, offering the possibility of documenting authentic acts of conceptualization, problem solving and expression.

Assessment is like telling the audience what the note is -- reference to assessing music.

Evidence-centered design: A design for any digital media learning experience...with computational representations of assessment functions. How can we make this entire thing as computational as possible in digital media environments?

Decisions and Dilemmas:

• Four part architecture: Mislevy Model details. Student model, assembly and delivery methods, interpretation method, observation situations. Low prob of being able to predict, (true and false test). Assembly model is more like SAT, because if students gets question right, no need to ask more about that topic.

• New media affordances:

• Decision and dilemmas. Purpose (reflective purpose, sonnet, mirror, map); audience (personal, trusted others, publics); artifact (media, ownership, focus),: (3D framework of decision-making about ePortfolios). 

Maximize your decisions about what you're doing, and decide which of these is the most important for your audience. You cannot maximize all of these features for assessment.

Intersectionality will be a really important concept in digital media assessment, it illustrates the need to understand. Complexity analysis is going to be helpful in dinging coherent resolutions among the layered an doverlapping frames of reference. Venn diagram of Author, experts, and editors. An example of peer to peer assessment (http://www.nytimes.com/2010/08/24/arts/24peer.html? )

New media affordances: complex performances, long-term records of documention, increased capability of eevidence of expert-novice differences. Entire system to be more real-time. Emerging capabilities in metadata.

Example of metadata generation: automated tagging and text mining. Feed in some documents or web pages, not to replace human part, but to complement and support. Semantic proxy demo. Looking at a URL and then generates tags.

Machines will learn to make their own topic maps by observing performances and artifacts, and will be able to compare their topic maps to those in order to make an assessment. Different than concept maps. Will include pattern recognition and judgment assistance for humans and computer agents. Another example: computer recognizes similarties between two visual examples, eg same color and same cone-like shape. More like a mental map. How can the machine-world become a helper to us as we try to make more complex judgments of assessment.

imagine a GIS map of social characteristics. Used this to answer: How do student grades relate to where they live? They look at GIS map of grades based on schools based on where students live. There is a connection.

We're looking at expert community and the way that they give out badges based on performance-rubric for performing in the community, based on how you give feedback.

Potential for having machines playing out underneath learning and assessment in something like Second Life.

Cathy Davidson is now moderating questions, and asking her own: If I were an educator who was asking, is there a better model for the digital age? David: Many pieces, different for each scenario. But, politics, mechanics, and the theory, which has been fully ingrained for 100 years, are entrenched and won't be easy to budge. What I have shown are pieces and parts of that puzzle. We need to develop Digital Media Assessment Theory. If it isn't scalable, efficient and affordable, it won't get done. We are at the brink of seeing some of these pieces come together. Game and simulation folks may have many of the answers for this.

Julie Keane: Individual teachers are now looking at these tools individually, applying them on their own. They are held accountable to other tests, but they are also held accountable to 21st century learning skills too. David: 21st century learning skills will push the door open, because the current testing standards can't get at assessment of 21st century learning skills. We know what the problem is, we know what some of the solutions are, but we don't have the package yet. Human-assisted machine-scored stuff <laughter> will be the way we go.

Question from the audience: what about surveillance? We can see so much of the personal space, so what are the ethics there? David: be aware of what that private/public boundary means, a topic that belongs to decisions and dilemmas. Should we be doing this here? Not to be done in an environment where the student doesn't know they are being evaluated (but David points out that Google already does that).