Human in the Loop (Being in Data)

The commonality between science and art is in trying to see profoundly – to develop strategies of seeing and showing. – Edward Tufte

Conceptual Sketching

in the Digital Model

An LTH / Bentley Workshop in London, March 18-20, 2015


Conceptual sketching is a way of seeing and showing. In a real sense, seeing and showing is thinking.[1] Sketches, and the thought they express, may now be used in new ways, presented in-situ within a “reality model” (within a model composed of many media types). Locating sketches this way may amplify their power to guide the development of design, and guide the development of communications describing what is to be understood and what, ultimately, is to be done or built.


figure 1: Click to play YouTube video ; model and drawings by Mark O’Bryan[2] , software credit[3]

[1] This idea is recurrent in the literature of the study of mind. An excellent example is this book:

[2] Mark O’Bryan, Architect,  Focus Devices in Hybrid Data: Usefulness in Design Thinking and Communication, page 300-305, link to paper

[3] MicroStation Connect Edition, Bentley Systems, Bentley Innovations Portfolio 01, and 02 and 03.


Contemporary media for communication and creation has led to the advent of the hybrid model; a model that celebrates diversity and combines various media types in a single environment. With increasing richness, this type of model also grows in complexity and to remain meaningful, a diverse array of “focusing devices” within such models has been proposed, the purpose of which is to show within the data environment what to look at, or, what to understand, and do. Focusing events in the data, provided by various kinds of focusing devices, provide articulate communicative focus. One type of such a device for doing this is the literal instantiation of conventional drawings in-situ. This has been implemented in commercial software, by Bentley Systems, since 2012. Examples can be seen here 01, here 02, and here 03.

Such techniques of guiding viewers toward what to see do not enforce a singular way of seeing, and do not force everyone to see the same things. That would be extreme, on the opposite pole from the other extreme which is to set people loose in an environment with no expectation of what they should see, other than that they should wander around encountering things randomly without even the possibility of guidance to anything specific.

The middle ground between those two extremes is the reasonable ground: people are free to roam an environment, and may be guided to events worth their attention, and may in turn detect and author focusing events and guide others to those. The methods for doing this are going to be diverse. Note this example from Microsoft’s Hololens demonstration, at about 1:15 in the video:


Figure 2: Screen capture from Microsoft HoloLens demonstration

This Mars demo is a clear example of this middle ground, of being drawn to something in a data environment, and at the same time being free to roam looking at anything. We suggest something else here too. One may have the sense of being in the data, perhaps of being “the human in the loop”.

It’s worth dwelling on the connection between “being in the data”, and being drawn or guided to specific events in the data. We suggest that those two things are very closely related.

René Descartes doesn’t offer “I think therefore I am” as a proof of existence. This is the common mis-take, misreading of it though. The stronger reading is to read “Cogito ergo sum” as definition: we are beings that think; this is what we are: thinkers. Whether we exist or not, our being is thinking. Heidegger picked up on this (Thinking is Being) and carried it further (Casein). See further discussion of Descartes here: Conjunctions

So, knowing this, that being is thinking, then of course “being in data” (or Heidegger’s “being in the world”) is “thinking in data” (or thinking in the world).

Je pense, donc je suit.”, and, I am, where I am; which is in the world, in data. And of course I can’t be (can’t think), other than in a world (in data)

Alright then, so what is thinking? Seems a big question, but let’s narrow it down and try to define it. Go again to the connection between “being in the data” and being drawn or guided to specific articulate communicative events in the data.

In the literature of the study of mind, linguistics, and artificial intelligence. “To think” is to select and narrow down from everything, to what matters, to first of all detect what matters in the environment, and to “conceptually encode” this in a manner that supports understanding and subsequent action. As Douglas Hofstadter and Emmanuel Sander say in Surfaces and Essences (simultaneously written in and published in French and English): Intelligence (thinking) is the ability to narrow down from the avalanche of information in an environment and “put one’s finger on what matters, to find the essence of a situation”.

Inspired by Hofstadter and Sander, to think is to act, through some kind of technique or another, within the world, within data, in such a way that produces understanding. This is usually (always?) motivated by a need to act appropriately within an environment.

In support of appropriate action, a thinking being selects from from among ALL available information/data in an environment; it detects and decides which data matters more and which data matters less, which data matters a lot, and which can be ignored, for whatever purpose at hand. To do this indeed is to put one’s finger on the essence of a situation. As Hofstadter and Sander say, that’s what thinking IS, putting one’s finger on the essence of a situation.

We use technique to bring things in an environment down to size so we can grasp, make sense of them. Without such technique we’re overwhelmed by data, by the world; we’re dysfunctional, can’t function, can’t act, can’t think, can’t therefore be.

But let’s narrow down and focus a little more. THINKING is the INTERPLAY between ALL THE DATA in an environment, and the act and technique of focusing, narrowing down so we can grasp. Understanding grows in the fertile ground between the WIDE TOTALITY of an environment of information, and our act of thinking, narrowing, focusing, articulating. It’s a back and forth.


  1. narrow down, from the entirety of all data around  us
  2. select and focus on what matters, in support of some appropriate action
  3. declare what matters, to oneself (at least) and to others
  4. articulate what’s declared in such a way that renders what matters intelligible, graspable, understandable, actionable

If we’re going to “be” in an environment (in data), we’re going to have to “think” in data, and here perhaps we can see what that means. To be in the data (in the environment), is to think in it, which is to find what matters, to develop articulate communicative focus to see and share what matters in ways that can be understood.



This definition of thinking has its analog in a typical set of drawings. When you look at a set of architectural or engineering drawings, what they are doing is quite literally 1, 2, 3 and 4 above. Each drawing defines a “here”; affirms each “here” as sufficient (what should be here, is here) for understanding something worth understanding, and presents what’s affirmed there in such a way that renders it intelligible and actionable.

We see the possibility to move beyond “having data”, toward “being in data”, “thinking in data”, in myriad ways. The techniques, tools, and methods supporting this are nascent, but already compelling. We propose to treat data as a substrate within which thinking can and has occurred and is still visible. An environment within which to narrow down and focus, draw attention to, clarify, assert, articulate what matters such that the avalanche of raw information is rendered intelligible and actionable. This puts being, thinking, working, communicating, in the primary position while de-emphasizing the pursuit of data perfection which is often unattainable, and today routinely counterproductive.

The movement of sense-making devices and techniques out of the abstract and into digital environments in-situ is a recent new development, that’s surely just at its beginning. It holds the promise of profound transformation of the potential of both the techniques of sense-making, and the digital environments they make sense of. A frontier is in view.

This brings us to the subject of the workshop: Being/thinking IN data, and being there when both the thoughts (in the form of sketches) and the environments through which those thoughts reverberate, are just coming into existence, when the thoughts and the environment are nascent. We’ll focus on the expression of design thinking through sketching and the possible amplification of these thoughts as they influence or reverberate into and through modeled environments during early stages of design, when thoughts are just bubbling up, when ideas are not fully formed and are being formed, and the environment is in formation, its form as yet, unknown. We want tools and media that help us discover, unfold, and develop thought, and form compelled by thought, as we exist, as we think, IN the environment.

Workshop Program

We’ll sketch and use new innovations in software that seek to amplify the power of sketching. We’ll test the idea that sketches presented in-situ within a model can guide design development, and guide the articulation of communications describing what is to be understood.

Architecture students from Lund and London will meet in London to exercise their thoughts. We’ll see what unfolds, and how it unfolds. Perhaps we’ll raise awareness of our individualized design process and thinking conventions. We’ll seek new insights about future developments in media supporting design and thought, while speculating on software innovations supporting this.

Before the Workshop

  1. Develop a hand built model
  2. Develop some hand drawn sketches

Sketch in either vector or raster or both. My preference is to emphasize raster at the beginning. Sketch by hand, Unfold, develop, and express thought (think) about the model in plan, section, elevation, and perspective sketches. The sketches are loose, need not be precise; they are conceptual and exploratory; they will evolve. Example:

Give some kind of indication of the relationship of the sketches to each other. For example, if you have section sketches, put callout symbols on a plan sketch showing the location of the section sketches.

These first two items can be done in tandem alternating between 1 and 2 as you develop them. Or you can do 1 and then 2, or, you can do 2 and then 1.

  1. Convert the handmade model into a point cloud

Use either scanning, or photo conversion. For small models, best results may come from photo conversion. There are quite a number of available free online services for converting photos into point clouds. Use any you like.

  1. Convert the hand drawings to raster

Simply photograph the drawings. Take some care, using any method (pre-photo or post process), to maintain some consistency in “scale”. That is, I mean, within reason. Certainly we can do image scale adjustments later as needed. Just be aware of the logic of getting drawings of consistent size into one full scale digital hybrid environment, again as this example:

At the Workshop

  1. Load the sketches and the point cloud of the handmade model into MicroStation

We will show the procedure for doing this. It is simple and follows the standard logic of producing drawing views from models. Your raster drawings will simply be attached to digital drawing sheets. Because of a unique function in MicroStation, all graphics from the sheets, including your hand drawn sketches, will automatically appear in-situ within the model, like again this example:

  1. Continue to develop your sketches

As the thinking evolves over the 3 days, those hand sketches can be either kept as raster, or, replaced with vector (which may include some tracing in place), and the use of those vectors to push a model forward. Iterate the sketches by any different possible means:

  • sketch again from scratch and replace (use same image file name OR new image file name)
  • sketch on a sketch, and replace (use same image file name OR new image file name)
  • simply edit the sketches in photoshop and save with same name (changes will show up in place in the model) Use any graphical editing techniques that help you uncover, unfold, explore and develop your ideas, whatever they are.
  1. Observe your sketches on their own, and, in-situ within the model

Consider whether or not seeing them in-situ in the model shows them in a different light so to speak. Does their presence within the point cloud of your hand built model bring new ideas to light? Does it help you somehow? Is it possible that new thoughts occur and new paths to explore are found? If so….

  1. Pursue your ideas

Continue unfolding and developing your ideas by iterating the sketches. Put yourself in a loop through items 6 and 7 until you think you have something.

  1. Use the in-situ sketches to guide the development of a model

Extend beyond 6, 7 and 8. Start building a model inspired by the sketches. If you like, trace some aspects of the sketches with digital vectors. From the vectors, curves, start driving modeled form. We will show a simple set of tools for sketching and modeling in MicroStation along with the controls shown in, again, this example:  Develop at least 3 iterations of the base scan data that are informed by discoveries made using software and novel modeling techniques.

  1. Documentary

Record mobile phone video of the workshop, projects in process, things going on. Collect for use after the workshop. Record findings at the end of each day and write, or put into graphics,  a short conclusion from the day’s process.

After the Workshop

  1. Documentary

3 Comments Add yours

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s