Studio|Research Methods Log

MDes Semester 2 : 2018

40 min readJan 23, 2018

Week I

I was disappointed to hear that the class which hosts the Microsoft Expo is not being led by Microsoft. This is a big loss for my class in terms of exposure to industry professionals and guidance from leading professional designers. This was one of the core elements of the curriculum which I came to this school to experience.

AI is a topic that I love and I’m eager to get into it. That said, I also have no domain knowledge and I am concerned that this project might become a bunch of hand waving called AI.

I’m happy to see that someone put forth Sam Harris, I listen to his podcast and I think his idea that we will inevitably one day create mechanical intelligence that exceeds our own in every way (AGI), and we should therefore address the control problem as soon as possible is correct.

I am also concerned that large tech companies have a way of working which is antithetical to the required caution and safety engineering mindset that we should be taking with AI research. Companies such as Google, Facebook and Twitter have demonstrated a willingness to rush flawed, not fully understood products to market as part of a MVP strategy. This is questionable in web development, but with the advent of AGI this strategy could be a complete catastrophe. But I don’t see how we can avoid it, as companies are incentivized to get products to market before competitors so that they can enjoy the maximum marketshare.

My team got started discussing ideas and what we are interested in, and disagreements started immediately. We have a variety of interests and visual styles, so this may be a challenging semester.

Week II

Ryan Baker spoke about data mining in education. This was a useful orientation to a particular family of AI interventions in education. Ryan has solid, evidence based explanations of what AI is and what results it derives. I enjoyed his scientific nuance in describing where we can be certain, and where questions remain.

APPLICATIONS

Dropout | Success prediction
Engagement
Student emotional state

ADVANTAGES

What has been achieved in academic projects
Currently outstrips what is available

INFERENCES

Is the student learn to solve complex problems that require inquiry?
Is the student developing rich conceptual understand in domains such as physics?

ROBUST LEARNING

Will the student remember what they learned?
Is the student prepared for future learning?

DISENGAGED BEHAVIORS

Gaming the system
Carelessness
Inexplicable behavior

EFFECT

It is feasible to infer these facts solely from student interaction with the learning system
Although using sensor, where feasible, can increase model quality, privacy concerns prevail

EFFICACY

Leads to better learning than traditional homework
Leads to better learning than traditional classroom practice
Use data mining to find behaviors which co-occur with human observations
It is best to use theoretical understanding AND mechanical detection

ATTENDANCE

Apply detectors to attendance data
Detectors can predict whether a student will do to college or not
Forum lurkers are just as likely to submit a scientific paper as forum posters

Team Presentation

We are making an initial presentation of our team territory map. My team considered the domains of health, refugees, language and culture. After some deliberation and intial contact index creation, we decided to go with refugees/immigrants.

Vikas gave me some useful guidance in creating our territory map, which was my assignment. He encouraged me to make an affinity diagram of our stakeholders and challenges, creating logical subcategories, and allowing the form of the territory map to emerge from that exercise.

Civic, Service, Basic Needs, and Culture were the categories that emerged

From this exercise, I sketched out various 4-lobed shapes which would allow me to convey some of the relations and connections between stakeholders and challenges. Overlaid on top of this diagram, I put sections which indicate groups of stakeholder and challenges which represent opportunity areas in health, local customs, and civic rights.

Week III

Marsha Lovett spoke about combining personalized education with data driven insights in order to help students and teachers learn together more effectively.

I thought her presentation was insightful and her work is clearly valueable to teachers who are looking to undersrtand and notice more about their students. From the student side, I wonder about how applicable her approach is to a variety of fields. She presented many clear and excellent examples of how data driven insights can help students perform better in mathematics and statistics. However, it was harder for me to understand how data, which is inherently a quantitative measure, can be used to provide insights about qualitative subjects such as writing, visual reasoning, spatial reasoning, language and social interaction without resorting to massive human time costs and a large amount of fudging.

That minor critique said, I really enjoyed this lecture and an excited about the potential of technology to help students and teachers alike.

INTRO

Individualized teaching is more effective
Teaching must occur at scale
This can happen in the same system

GOALS

Capture as much data as possible, clicks, opens, eye tracking
How individuals cluster together and how they are different
We are trying to create predictions with algorithms
But we need explanations and diagnoses in addition
Combine data with the scientific model of learning

LEARNING DASHBOARD

Model based approach does not just predict
Provides targeted feedback and proposed pathways to success
This helps students be more independent, self reliant learners
More knowledge acquired for less time invested

SYSTEM PROPERTIES

Questions have progressive hints and interactions which help quantify the student’s understanding
The system continually assesses the student’s performance
Model of technology relationship : Substitution, Augmentation, Modification, Redefinition
Technology becomes a targeting device, telling teachers when one student understands and another still needs additional instruction
Helps teachers to understand which sections of their course are successful

ISSUES | CHALLENGES

The majority of teachers use technology to reinforce and sustain existing patterns of teaching
We need to collect data which not just easy to collect, but genuinely useful.
Just because something is easy, does not mean it is a good use of technology
Lecture capture
Limits teacher
Discourages Q|A

Week III : Group Progress

Our group has been stuggling to find consensus on a topic of interest and we had some interpersonal issues to work out. That said, now that we’ve worked things out I believe things will improve significanlty. I am very interested in conflict resolution and team dynamics so this has been a great learning opportunity for me.

We decided on the name team whaleshark, and I drew a logo for us for fun:

I am working on my team facilitation skills, and I notice that when team whaleshark relies on verbal discussion, we often have miscommunications and disagreements. However, if I respectfully lead a communal white boarding exercise, things go much better.

Peter provided some very helpful resources on team conflict resolution which we have been applying to come to agreement. So currently the team is a little behind, but we are unified and working smoothly. In some ways I think it’s better to have these disagreements early.

Week IV : Austin

Former professor Austin Lee tuned in with his Microsoft team mate Jae Park, and they gave a very enlightening talk on AI. I found Austin’s perspective practical, thoughtful, clear and actionable. I particularly found his description of the future human-machine symbiosis-cooperation inspiring!

INTRO

AI and the designer’s role
Designing AI
Design FOR AI
Design WITH AI

POP CULTURE

Films : emotion
Is it conscious?
Loss of jobs?

TECH LEADERS

“I’ve talked to Mark about this topic … his understanding is limited.”
WW 3?

AI

AI is computers doing things that we think of as intelligent
Machine learning, NLP, expert systems, vision, speech, planning, virtual assistant, conversational user interface, in context suggestion, face recognition

DESIGN

Motion for status feedback
Voice input, NLP?
Type
GUI
Natural user interface (sensors, camera, facial recognition, gesture, voice)
UX for machine learning
Microsoft Azure (future data predictions)
Google DialogFlow (make a virtual assistant)

ETHICS

Racism
Gender bias
Biological parochialism
Value of various human lives

PRINCIPLES

Humans are heroes
Balance EQ and IQ
Honor social values
Know the context
Evolve over time

AI | HUMAN

What do humans do better? Top down perception
Where will AI always beat humans? Bottom up perception

PRODUCTS

Adobe Sensei
Autodesk Dreamcatcher
Microsoft Lumen

ETHICS

Everyone will design
Everyone will have powerful design tools
Curators rather than creators
Storytellers, multi-disciplinary creators of design tools
Superstar designers
Increase in AI productivity
Traditional to virtual design
There is no hard differentiation
Designing the future of humanity
Human empowerment

QUESTIONS

Allows you to spend time on things that matter
You can create your own tool to make things in exactly the way that YOU want to
Create your own tool, rules, approach and aesthetic

Week IV : Interviews

I interviewed Kristen Hughes, Terry Irwin, Mark Baskinger and Tom Merriman. Professor Merriman was particularly helpful: his Save Hayes Woods campaign is a case study that we can use as a model for the social action and advocacy that we want to spark with our product.

From the conversation with Professor Merriman, we identified the above local and national organizations to contact.

Week V : Interview

Jesse facilitated contact with Peter Wray from 350 Pittsburgh, an organization which supports local and regional environmental campaigns. He was extremely helpful in laying out the local and regional landscape of stakeholders, dynamics and interactions.

We have made contact with the Sierra Club, Penn Future, 350 Pittsburgh, two on campus organizations and the Pittsburgh Parks Conservancy, so our group is moving forward well. I am concerned that we may be interviewing them starting next week in many cases which will make our presentation difficult.

Observations from Peter Wray

Top down vs bottom up organization strategies
Companies that have public relations concerns are easier to have leverage against.
Engaging the public is a main issue. Building a grassroots people’s movement.
Campaigns hinge on funding and government relationships
Divestment is an interesting path forward. Reducing the percentage of endowment money invested in fossil fuels
Mayor’s climate action plan needs monitoring, that is organized by the Sierra Club
Unions such as the Boilermakers are interested in job creation and see it as the responsibility of companies to take care of environmental issues, they trust the companies

Questions and Directions from Peter Wray

What role do faith based organizations play?
How can the practices of funding organizations be changed?
A campaign around greenhouse gas emissions and air pollution
How can coalitions or informed citizens be organized faster?
Many organizations greenwash, and the effects of this are measurably good, what do we do about that?

We also did a workshop with Peter Weeks from Phillips on design research. This workshop was extremely helpful in giving real life examples and context to how individual research methods can actually be deployed.

We did a simple exercise in teams and our team’s results were surprisingly useful for how little time and effort it required. One of the tips that Peter gave was : don’t spend a huge amount of time mapping and diagramming trying to make it perfect. Instead try lots of little exercises and extract key insights or observations from them.

Affinity sorting for interview information

WEEK V Doris Papanek Guest Lecture

INTRO

Curios
Design and Learning Network
Empower creative problem solvers

CREATIVE LEARNING

Ask good questions
Sort information

PROCESS

Gather info
Define problem
Build understanding
Show understanding
Reflect

BIG IDEAS

Audio engineering is critical to success
Democratization of learning in order to close the employment gap
Learn how to live in a democracy

DESIGNERS

We change people’s mindsets with each new product that we make
“Designers must take responsibility to ensure that peoples’ needs have a voice in the development of technology and the decision making process.” Victor Papanek
Technology is a tool not a goal

I enjoyed Doris’ willingness to state that designers have a responsibility for the products and services that they make. Too often it seems to me that designers are able to offload responsibility for their work to a company or manager.

The difference between craft and production is an interestingly different take on the craft vs. design split that we discussed in Cameron’s class, from Chris Alexander. As designers we can contribute to one or the other of those systems of production, and we bear the responsibility for what we choose. I wonder how designers at companies like facebook and snapchat think of themselves and the work they do.

WEEK V Dr. Abert Presto

Jesse, Mary and I went to interview Dr. Presto who provided many insights

INTRO

Works with CREATE lab, not the same
Studied chemical engineering
Atmospheric chemistry
“I’d love to get to the point where we’re pushing constant predictive update of what the air quality will be in their area.”
Randy Sargent CREATE LAB
Illah Norbosch CREATE
OPPORTUNITY: connection to community organizations like PennFuture, they help the scientists find homes to host their sensors.
OPPORTUNITY: They have had to canvas to find install points for the sensors
Shale cracker, marcellus shale
Marcellus action

AIR POLLUTION TRAVELLING MAPS

Air pollution mapping is useful for outreach because it’s visual
Outdoor air pollution estimates
Black stuff that comes from a bus
Allows people engagement because they can look up their own address per pollutant
Map is an interpolation of 70 sampling points
“When people see these maps, they immediately look for their own address.”

AIR POLLUTION DISTRIBUTED SENSOR

Low cost emplacements/devices, which are multipollutant monitoring stations
50 devices
Idea is to make a public web map from the distributed sensing system
Potentially they can be globally distributed (San Juan)
The scientists do all of the install, maintenance and operation

STRATEGY

It’s important to stay at arms length as a researcher.
In general I package stuff and hand it off to PennFuture or other organizations
Making animated, or interactive documentation (maps, photos and videos) from the CREATE lab
Heinz Endowment paid for the maps and required that they present their findings to the community groups regularly
Green building alliance
City climate action plan:
Reducing emissions, how will it be measured?
CMU’s distributed sensors can be used to actually carry out the monitoring aspect of the climate action plan.
Opportunity: Connecting non-profits, scientists, and community groups to share strengths and cover weaknesses and generate coordination
GASP, PennFuture and other groups actively recruit for areas of sensor monitoring
Then Presto goes out to present to those people to talk about the work
Try to keep it understandable, “I try not to keep it too dry. I don’t really have the most complex visualizations.”

WEEK V Heather Sage from Pittsburgh Parks Conservancy

Heather, the current head of community outreach at Pittsburgh Parks Conservancy, originally worked on the Save Hays Woods campaign with Tom Merriman, apparently as her first task at PennFuture (!). Her perspective and depth of knowledge were inspirational.

I travelled to South Side to interview her in her new office on Carson street. At this office, much of the government outreach and community outreach divisions work. It was a great trip.

She gave us a description of what works for coalitions and how they break down, as well as introducing us to Parks Rx, which is a cool program that connects parks, schools and hospitals in order to expose children and hospital patients to the health benefits of Pittsburgh Parks.

Magic device gear and Liz Sanders, I’m excited to try some of this out!

WEEK V Liz Sanders Make Tools

Liz talked to us about co-designing at all phases of the design process. This workshop was helpful in seeing lots of concrete examples of how to create generative workshops. I can much more easily imagine what we would do for our team workshop now.

INTRO

Design is an inquiry into the future situation of use
Co-design is the collaborative exploration of future situations of use
Co-design can appear at any phase in a project, but it accomplishes different objectives in different phases

WORKSHOPS

Movement of the body is critical for creativity
Materials can foster creativity
Experience is a unique and uncontrolled phenomenon, everyone will have their own
Make < > Enact < > Tell is a constant loop
Are you having people work independently on their own toolkit? Or independently on a collective set? Or working in teams?
Working on round tables allows people to work as groups.

WORKSHOP DESIGN

Selection and design of the workshop materials is critical
If you can pilot things that helps a lot
Get advice from people who are in the field or experienced (tom? Heather? Peter?)
Put everything up on a board, look at it and decide what you want
Create 3 times more material than you intend to put in the workshop
Allow people the ability to
Have people create their own persona by indicating what they should fill out, and have it be about some sort of directed ideal (NOT THEMSELVES). Create a persona about waste water use, the person who is perfect and the person who is terrible. This allows them to admit stuff without being ashamed.
Creating matricies, grids, and delineated spaces which are good/bad, desirable/undesirable, part of experience/not part of experience allows people to get started.
Deputizing group members to explain what they did and why throughout the process is CRITICAL
The rule is, look at what we’ve provided, read it, use it if it makes sense, always feel free to create your own words and imagery.
It is critical to provide both words and imagery

Team Whaleshark planning our co-design workshop.

I drew up a general outline for our workshop, which included multiple mapping exercises, a magic device exercise, persona creation and lots of sharing:

SECTION 01 | INTRODUCTION + ORIENTATION [10 minutes]

Explain who we are, what our team mission is
Explain what the workshop is and what we intend to do
Split participants into groups of 4 or 5 (we will assign the groups based on occupation and prior volunteering experience)

SECTION 02 | PERSONAS [15 mins]

In teams, create two extreme personas
One who really cares about the environment
One who does not care about the environment
Both personas are non-volunteering individuals
Persona templates will be provided

SECTION 03 | MAP PAST, PRESENT, AND FUTURE [15 minutes]

Teams will collaboratively create a timeline focusing on the journeys of the personas they created in previous activity with provided images, words, paper cutouts and their own annotations:
What has happened
What’s happening now
What’s going to happen
Focusing on how the actions of these two personas affect the environment and their communities
The timeline will include a desirable and undesirable future vision
Timeline templates will be provided

SECTION 04 | MAGIC DEVICE + INTEGRATION [30 minutes]

Teams will create a “magic device” that would make the act of learning about environmental issues and volunteering easier in an ideal world/scenario
Materials will be provided including buttons, dials, knobs and labels, teams will design their device.
The device should include a communication and connection aspect.
Based on the magic device they designed, team will rearrange, expand, and add detail to their timeline map in order to integrate their device into their timeline and demonstrate how the device changes the journey for the two different personas
They will create a trajectory which shows where things will go from where they currently are.
They will indicate what effects the device will have which will support or encourage this trajectory.

SECTION 05 | FINAL TELLING AND WRAP UP [20 minutes]

Teams will present and explain their work, including:
The overall timeline
The device
Their desired trajectory
The device’s role in this trajectory
Talk briefly about their experience of doing this work

I am taking the lead on the magic device portion of the design research workshop. I am excited by the possibility of co-designers suggesting both higher level abstract emotional needs AND direct functionality. In the coming week I am going to complete the step by step details of how the participants should create their devices. I hope the exercise will create an experimental atmosphere where we can learn together.

Week VI : Norman Bier

Norman was very enthusiastic, impassioned, thoughtful and realistic. I enjoyed his strategic and honest assessment of why things are not working, and what things are fake. I had trouble understanding how his high level analyses, however accurate, could be applied to our studio project, but I enjoyed listening to him.

Higher Ed challenges

Diverse student population (background and economic status)
System was not well designed to deal with this diversity
This degree even more important in the workforce
Income shrinks for non-degree jobs

Herb Simon

Father of AI
Develops Cognitive Science to understand organizational behaviors to understand what “Intelligence” is
Great CMU research does not get the exposure it should
Simon Initiative was created to promote these things

Learning

Students are poor at judging what they’ve learned and when they learned it.
Liking is not learning
No good ways to identify when we’re learning
We only have so many mental resources to apply to a task.
We can only learn or monitor what we are learning
Experts can describe only 30% of what they know

Data Breaks Illusions

Learning happens “inside our brains”
Model » Observe » Model

Learning Management System

A good learning idea surrounded by half-assed management capabilities.
Funding challenges — what sustains the online systems
“Grant funding treadmill” it always runs out
Develop new technologies, but find ways to sustain it within a university

OLI

Build materials
Capture and share data
OLI’s goal’s shifted to better learning and goals as more people used their curriculum
Some online learning system algorithms are black box system
Algorithms can contain unconscious biases

Learning Learning

Deal with crazy options to learning about learning. Harder than rocket science
Break it down into smaller A/B tests to start to unravel all of this
Give educators better tools to give faculty more control over customizing education

Week VI Austin Lee and Jae Park

We had class with Austin and Jae. It was extremely helpful and concrete. Austin and Jae are insdustry professionals who have real experience with the subject matter that they are teaching and I found their exercises to be thoughtful, compact, and enlightening.

INTRO

What is AI? Pattern recognition, data processing, robots.
Everyone will have powerful design tools.
You will have to fight hard to maintain your status as a designer

JAE

Living with robots
Divorce due to an affair
How should the AI interact with the family?
Who owns the AI?
Male vs female desire for the truth?
AI Self Portrait
We express, express ourselves, and express toward media
How does AI express itself?
Data is what allows the AI to express
Analyzing a video clip
Exporting analysis to parameters of vectors in processing
Use a gantry to paint.
Self portrait

CORTANA

Voice for Cortana from halo is the system voice in MS virtual assistant
Do virtual agents need a persona?
Beyond human computer interaction, how does the computer facilitate human to human interaction?
Visual feedback is helpful in understanding whether the system is reacting appropriately

PROTOTYPING TOOLS

Cross platform considerations. How do you deliver a consistent experience on totally different hardware?
Awareness, invocation, input, acknowledgement, understanding (CONTEXT RE FLOW)
Action and answer
Dialog and disambiguation
No match
Task complete (acceptable or unacceptable)
If you ask to call Jae, the system should know to ask for disambiguation before completing task.

Whiteboard detail which I drew of our proposed system in A//J’s exercise

Week VII

This week we are prepping for our generative co-design workshop. I’m exicted to get my feet wet in leading a workshop. I was inspired by Liz Sander’s talk and I hope our workshop will give us insights on a way forward for our design.

Our workshop plan has evolved slightly. We decided to have the participants create personas, and to map where they and their actions would fit within the larger mapping exercise. This will provide us with interesting observations of people’s assumptions and biases hopefully.

This week we had a talk from Qian Yang on machine learning. Her explanation of the fundamental difference between the design school approach (adapt technology to human needs) and the HCI approach (find a human need for a newly developed technology) was helpful. I thought her call to action for designers to get a basic grip on statistics and quantitative analytical methods sounded right. I am surprised that our education does not include even a basic introduction to this way of thinking.

We tested two of our five workshop elements with a team of design colleagues and came up with a key insight. The team crafted a negative environmental persona and then created a timeline for them.

In the share step of their work, they explained that this person had circumstantial events in their life which created a higher level value of preferring short term payoff over long term reward. The connection between higher level values and relationship to the environment was an interesting observation.

Week VII Workshop Edits

We received feedback from testing that additional scaffolding might be helpful in each of the co-design exercises. So we added clearer and more extensive instructional componenets. The participants were concerned that when they had to select headshots for their personas, the only factors for selection available to them were age, gender and race. Therefore they suggested that photos of people who were performing activities would add an additional factor to select from, which will make workshop participants more comofortable.

Peter also gave feedback on Aristotle’s concept of akrasia. Applying a moral conceptual framework to our personas is an interesting idea.

Week VII Testing

We ran a test with Josh Lefevre and recieved postive feedback on our workshop, below are his outcomes.

Week VII Workshop

On Saturday, we held our workshop. We had 9 participants ( 5 design masters students, 1 undergraduate from the CMU Sustainable Earth Club, and 3 professionals from elsewhere). We had them work in teams of 3 and perform the following activities:

Create 2 personas (one that cares about the environment, one that is indifferent about the environment)
Create the timeline of these 2 personas
Design a magic device that would make learning about environmental issues and volunteering opportunities easier
Redo the timeline to incorporate the magic device and how it would change the 2 personas future

Our workshop was quite successful and I’m excited to pull out some insights.

Week VIII Presentation

We received great feedback from Arnold Wasserman. He mentioned Etienne Wenger’s communities of practice work. The term “community of practice” is of relatively recent coinage, even though the phenomenon it refers to is age-old. The concept has turned out to provide a useful perspective on knowing and learning. A growing number of people and organizations in various sectors are now focusing on communities of practice as a key to improving their performance.

He also mentioned framing, which is one of the main things that designers talk about. So there are two layers of framing that we need to consider. One is how our group is framing this project. The other is how we are teaching our users to frame. For example, this is critical for our fifth insight “connect peoples’ personal values to a larger environmental context.” Helping users to frame and reframe flexibly will be key to our success so we will add that point to our learning principles.

Week VIII Learning Diagram

Susan Ambrose’s “aiding motivation” diagram served as the foundation for our approach to learning. Her 5 principles of learning are also helpful: Understand what kind of learner you are:

Approach the experience as an exploration, not a risk.
Start small: Concentrate on the components first, then play the game, sing the song, join the conversation.
Don’t go it alone.
Remember that learning takes time.

The learning approach that we have specified recognizes that expectations and values are critical foundational factors in a user’s motivation. Most of our interventions center aroudn this phase of the learning flow.

Week VIII Concepts

We generated 7 concepts and then selected the best 3 which are shown below.

Week IX

This week have been clarifying our concepts. We realized, from helpful student and professor feedback that our concepts actually seemed like pieces of a larger whole. I went through the previous year’s presentations and I related to “The Real World’s” thought process. Their presentation shows a variety of concepts which they bound together as various functions of a larger MR system for auto repair technicians. I think a similar approach will be helpful in our group, many of our concepts seem like individual functions or aspects of a larger system.

We presented a unified concept which is a lifelong learning system for people who are concerned about the environment but do not know how to implement the practice of an eco-friendly lifestyle.

The system centers around a device which is a future version of an in-home AI assistant. Smart appliances and sensors complete the IoT home system which monitors and suggests more efficient, cost-saving, eco-friendly behaviors and practices.

Each home networked is then networked into a neighborhood level where community members can interact, request, participate and play games with each other.

Environmental organizations can put out requests and missions for community members to participate in as well.

Week X

This week our team is clarifying our ideas, scoping down and preparing for speed dating. We noticed during feedback sessions that although our idea is often exciting to people, our explanation often leaves them confused. Details about: the data, the system, the limits and protocols of the platform, and the intended change over time remain unclear.

In order to rectify this, we are going to create new diagrams and maps which explain in clear visual language each concept. This will help answer people’s questions and break down each concept into an understandable byte.

Week X Speed Dating

We feel pretty confident about an IoT home assistant centered around a physical device. However, our system is large and complex so we want to speed date specific aspects of our system such as:

Onboarding
Trust
Data collection settings
Contextual visualizations
Network interactions

Originally I started out hand sketching our storyboards which has been successful in the past. However, many aspects of our system are conversation based, so the sketches were not as successful as I hoped.

We re-drew scenarios in a simpler form and we will speed date with research participants in the next week before synthesis.

Week X Persona Exercises

In Bruce’s class we stared sketching out personas for our users as well as for our system AI Orion

After speed dating the major aspects of our system, we came together to perform a synthesis exercise. We wrote each category of findings on post its and then affinity diagrammed them into sensible categories.

From this exercise we created did a “yes/no” exercise is which we defined what the Orion in home system and community system would do. This was useful because it allowed us to determine illustrative edge cases which will help us create design principles.

Week XI Personal Interaction Self Evaluation

I have been thinking a lot about how my personal interactions with team mates can help or hinder team practice. In earlier sections of our work I advocated strongly for my ideas and asked many questions of others in order to understand and find weaknesses. However, I found that this generated a negative attitude in my partners.

My new strategy involves more listening and asking questions in a way which gives team mates more agency. I am asking things like:

Tell me more about that idea …
Why do you think that’s true?
Previously I heard (x) from you, now I think I’m hearing (y) from you, how do you see those coming together?
How do you envision this working?

Week XI Diagrams and theoretical frameworks

Peter pointed out that our network is complex and its objectives unfold over time. This can be confusing for people to understand, so having a theoretical framework makes our idea sound more important and simpler to grasp. We are using three different frameworks.

The Social-Ecological Model

States of Change Model

Week XI Concept Scoping

We had very productive meetings with Kristen Hughes and Jonathan Chapman who helped us scope and focus our design. We are removing a lot of the social and community building aspects of our system and trimming it down to a single clear objective:

Help humans understand their stake in the environment by expanding their definition of home to include local natural features, organizations and political events through daily home practices.

Week XI Product Sketches

Laura produced some beautiful products sketches to start getting an idea of what our form might look like.

Week XI Jonathan Chapman Interview

Our team performed an expert review with Jonathan Chapman who delivered extremely useful thoughts across the spectrum of practicality from pure philosophy to direct design feedback.

Interesting angle: easing people into awareness
An arc of awareness
Create a philosophical UX journey of home definition shift. Design those changes.
Think about this as a curriculum. What do you want people to know? Which ones build on each other.
Normal UX, how do you get someone there and then keep them there
We want something else: get people somewhere, and then gradually shift the goalposts every time until they get somewhere
MSC sustainability leadership at Cambridge, ready made information
Expanding a person’s frame of reference for where their responsibility begins and ends. In other words, expanding the definition of home.
Eindhoven, who do you give control of the house to? What height is the thermostat at?
Observe practices around control of the house. What do they indicate?
Any money you save is yours (child)
Social relationships: children and grandparents, what are people’s role? How can you leverage those roles for behavior change?
Nest, it’s exciting but since it’s passive and eventually fades out of your awareness
Nest is too polite, but you don’t want to be annoying
Wheels turning to indicate energy usage. How can something physical be used to materialize energy usage.
Make sure your system does not remove people from the daily physicality of energy usage
3 kids, 2 parents, massive house
Is it AI?
Energy Clock, a self-erasing archival data visualization
Physical representations and analogies for energy usage
Lamp that sheds a pool of light which is proportional to the energy usage in the house
Markings for averages with a current magnitude visualization
Numbers are problematic because they are abstract and don’t give an indication of usage
Social proof of energy usage
Design systems which allow people to behave in certain ways

Week XI Further Scoping and Speed Dating

Each team member has performed independent speed dating sessions on various aspects of our design concept. Last week we heard from Kristen Hughes, Jonathan Chapman, and a variety of other study subjects that the concept is going in an appropriate direction but needs to be tightened, clarified, simplified and streamlined. We have settled on an IoT device which monitors utility usage and air quality.

There are 3 phases of behavior change which our system intends to draw users through. First, most users purchase our product in order to make their homes more efficient and save money. In phase 2, the system introduces users to the local environmental features which create their utilities. In the third phase, users are connected to volunteer organizations and opportunities which can help to protect the sources of their critical resources. This three step process will lead to:

Expanding a person’s frame of reference for where their responsibility begins and ends. In other words, expanding the definition of home.

Our challenge for this week lies in achieving the following: synthesizing a list of concrete functionality, defining the role and execution of AI, unifying a visual style and AI communication persona, finalizing our product form.

speed dating material and synthesis board

Above are the materials for speed dating sessions: form variations, visual style propositions, and poetic communication explorations. This week we will test these with various study subjects.

Our visualizations have three levels to them. A resting poetic state which gives a direct and visceral understanding of the rate of usage. Beneath that is a numeric daily visualizations which is half-abstract half-concrete. Beneath that is a completely concrete visualization which shows week and month usage patterns. Goals can be set simply in this interface. Above are propositions for all three information streams.

The final consideration for this week is how our physical form will function. We are interested in creating a form which requires physical engagement from the user. We will provide the user with 4 objects, each of which is utility specific. Once per week or month, the user is intended to bring all four of these monitor objects to a base station which they can set goals at and access more complex in-depth visualizations. An open question: how do we incentivize the user to display this behavior? We have considered adding funcitonality which would allow the user to pay all of their utility bills at once, conveniently during this interaction. A less desirable solution: take away desired things such as electricity or water. Perhaps an audio cue.

Pictured above is our map with goalposts of the transformation which users will undergo in their journey with Orion. The map goes from “becoming aware of utility usage” to “taking action by volunteering”. This is a big journey, we do not expect all users to make it all the way through. The primary action of the Orion AI will be to assess where users are, what their interests are and how to most effectively move them along this timeline. Careful, contextual, well-timed interventions will be delivered through several channels of the Orion IoT home system.

The above map also demonstrates what learning principles and attention attraction strategies the system will use. Since the learning goals for each goalpost are significantly different, Orion must behave in a nuanced and multivariate manner to achieve this journey.

With 8 study subjects, we set out to identify what images people associated with the four utilities that we are monitoring. We found that water and electricity generated remarkably consistent imagery, whereas air quality and natural gas generated a wider diversity of images.

We also ask study subjects to map out their average day and indicate when they had free time to think about their utility usage//environment, as well as when they would be cognitively open to that topic. We discovered a “morning routine” opening, an “evening wind down” opportunity, and a “on the go daily commute” opportunity. Orion’s conversational and educational interactions will aim to fit into these openings.

Week XII Production Time

Our presentation went relatively well, as usual our group was asked a disproportionate amount of probing questions. In general I feel confident about where we are in the design process and that our product makes sense, is desirable and practical. We need to carefully define learning and the role of AI again so that our final is airtight.

My main task is to create animations which represent the poetic ambient visualization of utility usage. First up with be electricity, which is one of the more difficult of the four utilities. Laura created a beautiful rendering of the product which I will use as a base for my animation.

I am learning a new compositing technique to create a small ball of lightning inside the product. I am following Video Copilot’s excellent tutorial on lighting and procedural specular effects. The goal is to create an interconnected system of rendering effects which allow me to adjust intensity, color and diameter of the sphere with single controls each. This task will push some of the limits of my procedural animation capabilities which will be a useful challenge.

animating electric visualization and specular reflections

The specular detail in this animation is satisfactory, and is a basic proof of concept for further work. I will check with team mates for final design approval.

Week XII Electricity Visualization Demo

My objective is to create a video which demonstrates the full range of visualization that a user might see in their daily electricity use. To do this, I will create a simple counter in Ae using a basic script, and then coordinate the intensity of my visualization with this counter. The counter will range from 30 to 100 percent.

In order to make the difference between minimum and maximum use clear to a user, I will use the following visual variables:

Line weight of the lightning
Intensity
Length of lightning
Color change from gold to red

Combined, the smooth transition of these four visual variables should clearly show the difference between low and high usage, which is the purpose of this visualization.

typography, scripting, and lightning placement

After this task, I need to make this visualization look more realistic by creating specular highlights, reflections off of the deck of the device and light splash on the ground. The purpose of these details is to make the VFX look convincing by imitating the behavior of light in the real world. When designers don’t carefully include details like this in product videos, the effect is less than convincing which casts doubt on the product and underlying ideas.

masked ground splash and specular highlighting

Now that the demo is complete, we will need to test in order to assess whether this visual variable manipulation clearly translates to the meaning we want to see from users.

Week XII Team Animation Feedback and Testing

I showed the demo video to my teammates, and they said that the visual variables, although generally clear, could be tuned more. In specific, they asked if the representation of 30% cold be stronger, and the color shift more dramatic. So that’s my mission today: make 30% and the color shift stronger without taking the video out of the realm of a realistic look. I am concerned that this video will become too obvious and unrealistic, so it’s my job as a designer to balance the need for more clarity without making the video problematic.

The first problem to address is the fact that the 30% representation is too weak. This can be addressed by adding in two more lightning effect solids. This will double the amount of lightning forks represented. In addition, I can tone down the decay parameter, which will make those individual forks longer.

I experimented with renderings starting at .77, .67, and .57. The middle value is most appropriate and gives the lower end of this range a significant boost, while maintaining believability

The next issue is with the color and intensity of the red lighning. This can be adjusted by changing the color of the original lightning layers. Additionally, I have experimented with changing the scale of the central dot. Although this introduces an additional visual variable, I think it quite well emphasizes the effect while maintaining believability.

Unfortunately the video did not come out as intended, a lot of the visual information is fully burnt out from settings which are too high on the exposure effects. To combat this, I’ve set some new masking properties and reduced the lightning decay by 10% which makes the bottom end of the demo spectrum more intense

Week XIII Poetic Visualization Static Renderings

At rest, Orion displays a generalized poetic visualization of the user’s utility status. I have heretofore focused on electricity because this is the utility that will be featured with motion in our product video. The In the interest of saving time and conveying as much information as possible in our presentation, I am making static renderings of the other three.

Air Quality

The air quality visualization includes a rising column of black smoke and particles. This was based on imagery collected during our utility concept sketching research exercise. Visual variables include:

Height of column
Density of smoke and particles
Color of baseplate

This visualization does not give an indication of proximity to the user’s goal, because it is not possible for the user to set air quality goals. The red/brown/black color palette is threatening and invokes disgust to help the user begin to care more about air quality by making its danger visible.

Water Usage

The water visualization leverages the metaphor of a filled water vessel draining in order to help the user understand how much more water they can use before they have exceeded their goal. When Orion is full, that represents no water used, as orion empties it represents the decrease of available water before the goal is exceeded. Visual variables used:

Volume of water in the vessel

An aqua/light blue color palette was used to help distinguish between water and natural gas.

Natural Gas

The natural gas poetic visualization leverages the metaphor of a stove flame or camping gas burner which runs out of gas over time. At the beginning of the day, this visualization is a bright blue flame which reaches close to the apex of the Orion interior vessel. As the user consumes more gas, this flame will eventually shrink and go out, indicating that the user has consumed their maximum goal for the day. Visual variables include:

Flame height
Flame color (blue to orange)
Intensity of light

Study subjects in the conceptual utility sketching exercise consistently used the metaphor of a gas flame. This is fortunate for our team because gas flames have a brilliant royal blue color (which helps differentiate it from the water visualization) and a bright orange color which is associated with warning. Additionally, the flame burning lower includes the implicit metaphor of a gas volume emptying in response to use.

Week XIV After Effects and Teamwork Reflections

One of my main efforts this year has been to work on learning to be a better, more cooperative and more functional team mate. The dynamic on my team has been challenging at times, conflict is frequent and the delta between our points of view is no-trivial.

I have been using this context as a lab for testing and noticing different behaviors on my part to see if I can elicit a more functional dynamic from the team. In other words I have been focusing on the questions:

If there is a problem, what if I assume that I am the problem?
If someone disagrees with me, what if I assume that they know something that I don’t?
What if every time something goes wrong, my default position is to assume responsibility for it?

Working from the default position that everything is my fault when things go sideways is actually empowering. Not only does it focus me on a results oriented attitude, but it encourages a culture of responsibility on the team.

There is one major barrier blocking me from achieving this attitude on a daily basis : my ego. Progress as a team mate and professional will only occur when I have learned to appropriately notice my ego getting spun up, take a step back, detach, and suppress my egoic response. The first half of the semester I was generally unsuccessful at this, but it has become easier recently.

Week XV Final Video Editing and Asset Polishing

This is the big week, everything we’ve done is coming together into a 10 minute presentation. This week and the last have gone extremely smoothly for my team from a cooperation standpoint. Working on this team has been a special privilege, not only because my team mates are so talented and intelligent, but because they think so differently than I do and they are willing to express themselves.

Week XV The Whale Shark Experience

As a thought experiment, I will outline two common dysfunctional team dynamics in order to show why I am so happy to be a part of team Whale Shark.

First, the totally agreeable team: this team has people who all think precisely the same and want to do the same things. This team SEEMS great because every is so agreeable, but in reality this team produces boring and uninspired results because there is no complexity to their solution.

Second passive aggressive team: this is a team in which everyone actually totally disagrees with each other, but no one is willing to say anything. This team also SEEMS great, but actually produces poor results because one or a few members of the team actually believe in what they are doing.

Team whaleshark is different because we all disagree strongly on most things, but we are willing to express ourselves fully and have learned over the semester to keep our egos in check and let the best idea come forward, no matter who it came from. This means that although our team seemed slow and fractious at the beginning of the semester, our last 2 weeks have been extremely productive and successful and we’ve outperformed my expectations in every way!

Week XVI Video Editing and Volumetric Display Assets

Mary and I are splitting the video editing work and I am totally responsible for the volumetric display renderings. The challenge for the video was to insert a digital rendering of orion into a real scene and have all of the appropriate specular and reflective light properties be represented realistically. Fortunately, I spend MANY hours practicing this over the last couple weeks so although the task was a challenge, I was prepared.

editing the lightning composition properties

compositing the lighting effects into our video scene

We ended up choosing the water vessel rendering for the cover slide of our presentation and I am proud to have contributed in a small way to my team.

One of the main challenges that I faced was integrating realistic lighting effects into a model which lives on a white background. In order to accomplish this, a set of subtle shadows and glare effects was used.

Week XVI Final Reflection

It’s been quite a semester and I’ve learned a lot in this class. One of the primary ways that I have matured actually has to do with team work and collaboration. Much of this I have already documented but there is one more part I’d like to record.

Certain activities lend themselves to simultaneous collaboration. For example, a design research session with affinity diagramming is excellent for sharing with multiple people. However, something like procedural animation, or highly detailed visual design cannot be done well collaboratively. That said, it is still critical to be able to take feedback appropriately, repsond to valid critique, and recognize that multiple minds are more useful than one.

Sometimes, when I am being particularly egotistical, I will put many hours into an animation and then refuse to accept criticism about the design from team mates. This is ineffective and arrogant. Instead, I have learned to justify my design decisions strongly with logical evidence. When I have no logical support for my design decisions, then I look at the criticism as a helpful piece of information which can make my design better. In this way, the strongest outcome can be achieved.

It is interesting to wonder how this principle of self-questioning intersects with confidence. I find that I must have confidence and trust in my intuition in order to achieve excellent results. Creativity requires a courage and risk taking strength that requires confidence. However, as noted above, one must be willing to be constantly questioned and be wrong. How can I be totally confident, yet always question myself? This seems like a paradox but actually is not. The golden path which balances these two poles is logic. Is there a reason or meaning for each design decision. If not, even if it is beloved, it must be changed or eliminated. This ability to be strong yet flexible is the same which is required for science or advanced philosophy. Perhaps all human endeavors, at higher levels, have commonalities.