Nonlinear Navigation in Movies with Flowchart Aid

Nonlinear navigation in movies with the flowchart aid - Article


Interactivity has demonstrated its potential daily; on television stations, for example, viewers can choose from which movie to watch the next night to how they’ll influence the end of a program. However, in films for attitude capacitation – where every minute a procedure can vary for two or more possibilities – was not available. The trainee watched a never ending  of: “if this happens do that”. Computer science, coupled with graphic expression, provides new possibilities to deal with this problem. The interactor could participate more actively deciding which portion of the video to watch depending on certain prerogatives, needs or challenges, thus watching a movie tailored to their needs. This paper discusses the possibility of adopting a flowchart that reflects the several steps of one or more video sequences. Allowing the interacting agent to decide the next sequence of the video that he wants to watch based on his need or curiosity. The graphical expression thus used assists the interagent in navigation and allows him to better manage his time and study needs.

Key words

Interactivity, Flowchart, Film, Nonlinear


One of the great innovations introduced in the graphical interface of computers was the possibility of visualizing video sequences. Initially there was a great euphoria, as the introduction of this new media paved the way for the development of even richer interactive software. The volume of video material at the time was enormous, a result of the “explosion” in video production at the end of the last decade.

Producers of interactive software realized great possibilities, since the process of digitization (on this, see: Digitization of videos, technical aspects) of this abundant material was relatively cheap. This apparent advantage, however, has repercussions on a fairly conventional use of this medium. In fact, the video was presented in full, just as it was generated analogically. Not always did the video introduced really add value to the program. In fact, the user could have seen the sequence on his VCR, sitting comfortably in his favorite armchair, interacting with his remote-control. It was evident the kitsh appeal in the use of this new feature.

In fact, this phenomenon is not new in the history of communications. Cinema in its infancy was nothing more than a filmed theater. The humorous TV programs, until recently, reflected the same language of the radio programs, where the image contributed nothing. And this phenomenon is reproducing every time a language migrates from one medium to another. As Arlindo Machado stated: “… machines succumb to the hands of production workers who only fill them with ‘contents’ from previous media, repeating in new languages solutions already crystallized in older languages” (MACHADO, 1996).

The employee, a naturally occurring category, is a sort of ‘prosthesis’ of the machine (it is an extension of the machine, not the opposite), in charge of activating its mechanisms and putting it into operation, so that it can Program for which it was designed.In fact, we would need to question what video is today. According to Machado, video is more than a sequence filmed by a camera. It can be reworked, shredded, altered by digital means, software, or by powerful editing equipment that today does more than simply cut and paste parts. “Speaking of video today means putting yourself, first of all, out of any institutionalized territory. It is about confronting the challenge and resistances of a hybrid object, fundamentally impure, of multiple identity, which tends to dissolve chameleonly into other objects or to incorporate its modes of constitution. “(MACHADO, 1996)

This article discusses the actual suitability of video in the use of interactive media, using graphic metaphors for its navigation. It also discusses the technical aspects of video digitization and ends with a case study, in which the objects of this research were implemented. (Demo of the program available at:

The flow chart as a visual metaphor
Non-linear video browsing

One of the technical characteristics of the computer is that it consists of systems that allow access to any part of them with the same ease. By analogy, we can compare a vinyl record with a cassette. The first allows you to listen to any of your tracks without going through the others. However, to move from one point to another of a cassette tape, it is necessary to cover all the spots between them.

This computure feature allows to create non-linear programs, and this, in our opinion, is the great differential that this machine presents on the others. However, producers of educational or training programs rarely make use of this advantage. Trying to reduce costs, deadlines or even by not recognizing this feature, they simply restrict themselves to scanning the original media materials without adapting them to this new reality. The result is text, movies or any other media presented linearly on the computer. This migration does not add quality to the final product, and communication could continue to be done in its original environment with the same efficiency.

On the other hand, overly random programs tend to be confusing and unproductive. As Romero Tori has observed, “… navigation in hyperbases, involving a great amount of information and numerous consultation possibilities, in which the attention of the browser is divided into several concerns, such as: assimilation of content, operation of the system, orientation in hyperspace and research organization and its objectives. “(TORI, 1996). The interacting agent feels at the same time stressed, having to make so many decisions, and lost, because as he chooses a possibility of consultation, he leaves others behind and as he advances, he loses his orientation. He feels trapped in a maze like the myth of the Minotaur. Lucia Leão presents this metaphor brilliantly in her work “The Maze of Hypermedia”.

Although in some cases, the intention of the producer is precisely to make the interactor get lost in the “maze” of hypermedia, in general, mainly in works with pedagogical objectives, it is necessary to create mechanisms so that navigation is not confusing. More than this, the product must allow the interactor to immediately locate the point of interest, also making the product a performance support, a work of reference, just in-time.

One of the commonly used techniques was reported by Romero Tori proposed the use of maps and spatial metaphors in hypermedia systems. According to him, “… A more concise and visual format could map the information in images that help the understanding of the structure and relationships between its elements. If these images are geographical maps or spatial environments user involvement will be even more effective, since we are already accustomed to be guided by maps and manage objects arranged in space. “(TORI, 1996). Lucia Leon also warned of this trend: “… some CD-ROM applications usually include general schemes and overviews, with the aim to offer your reader the overall look. In the event that the application has been built in a hierarchical way, this mapping can help a lot. “(LEO 1999).

If we consider a map as a geographical chart, we can also see it as the spatial representation of a procedure. For when we draw a map of how to get to a place by hand, we are not only representing the environment, but also describing the procedure of how to get to the desired point. In this sense we can classify the flowchart as a special map case.

Hypermiditic interactive programs such as educational or training software generally describe procedures. Training videos usually also describe procedures, however, most of these procedures present decision point. At this point, it can range from two or more possibilities. In conventional videos – presented in a linear fashion – we watch a generally confusing film, one no more ending up “if this happens, do it”.

The computer, along with the graphic expression, provides new possibilities to deal with this problem. As stated earlier in this article, the computer allows you to access any data with the same ease. The video, when scanned and executed on the computer can also be accessed in a nonlinear manner. The interactor can assemble the video in the sequence that best meets his needs, according to certain privileges, needs or challenges, so watching a film tailored to his needs.

Earlier in this article we referred that in general, videos embedded in multimedia projects have a conventional presentation. The interactor viewing it from beginning to end. Buttons that mimic the VCR control are included in the interface, giving the false impression of interactivity. Again we see that the media has changed language without suffering adequacy.

An alternative, researched and implemented with great success for our group was to just take a flowchart reflecting the various steps of one or more video sequences. Although, most of people are not familiar with them, they have few symbols and their geometry is easily assimilated, almost intuitive. As we shall see, the flow chart showed to be a powerful resource for interacting to locate in a hypermedia system.

Scan videos, technical aspects

The purpose of this chapter is to provide the reader with a basic understanding of the operation of video playback systems and therefore does not intend to exhaust the subject. In it we will discuss the technical aspects involved in converting videos stored in analog format to the digital format or capture real situations directly for this medium. So if the reader is not interested in technical aspects, we suggest you move on to the next chapter.

A captured video in analog format is recorded in the form of “representation of numerical quantities, using continuous physical variables” (Meirelles, 1994). As its name suggests, the analog format compares an information with other generating a result that can have an infinite reason. (Figure 1).

curva digital - representação de sinal analógico de vídeo x representação de sinal digital de vídeocurva analógica - representação de sinal analógico de vídeo x representação de sinal digital de vídeo

Figure 1 – analog signal representation of video x digital video signal representation

In contrast to this mode we can record or convert video to digital format where: data is represented in binary form, ie “the” information is represented by a set of bits “(Meirelles, 1994). (Figure 1).

To simplify the understanding of the video conversion process between the formats, we will begin discussing how to convert a sound. Then we will discuss how to convert still images and finally talk of video conversion basics.

Sound Basics

.The term ‘sound’ refers to the acoustic phenomenon obtained by the compression and expansion, in wave form, of the particles of a physical environment, – air, for example. It “propagates in all directions, for example, the waves that form when we throw a rock into a calm lake” (Luthier, 1995). “When something moves in the atmosphere, such as the string of an instrument or our vocal chords, the air molecules move” (LINDSTROM, 1996), generating pressure variations. We hear these variations, when they reach our ears. “Human ears are a type of sensor or transducer because they convert the audible pressure variations into an electrical format used by the brain” (Luthier, 1995).

To record an existing sound in nature, we use a transducer – a device that’s able to convert one type of energy into another- to convert the kinetic energy of sound waves into electrical energy used by electronic equipment. The output of this process are the different levels of electrical voltage corresponding to different pressure levels change. “A microphone is a transducer example. Voltage levels, related to the analog signal, are then used to create a pattern on a magnetic tape. For playback, the tape signal is read by playback heads, is amplified and passed to the speakers. The speaker is another type of transducer that performs the opposite conversion to the microphone. “(LINDSTROM, 1996). The speakers convert different energy levels back into kinetic energy, recreating the original fluctuations of air pressure and making the sound audible.

Animation 1 – Quantization

We called the electronic sound audio signal. It “is measured in a two axes graph -voltage that varies with time” (LUTHER, 1995) (Figure 1).

An audio signal is also composed of other parameters: amplitude, frequency and period – or wavelength. The amplitude is the sound height and is measured in decibels (dB). “The greater the pressure exerted by the moving object, the higher the sound and the greater the number of decibels” (LINDSTROM, 1996). In electronic systems the amplitude is the wave height, and “is measured by the maximum and minimum voltage variation of the audio signal” (LUTHER, 1995) (Figure 1).

Sinal de áudio - onda sonora

Chart 1 – Audio signal

The second parameter for an audio signal is frequency. According to Badgett et al. (1994) we can imagine the frequency as being the amount of waves passing through a fixed point in a second. “Most audio signals periodically vary from positive to negative voltage and return to positive again. The speed at which this periodicity occurs is called frequency, and is expressed in cycles per second. The unit of frequency – one cycle per second – is called hertz, (Hz) “(LUTHER, 1995) (Graph 2).


Sound files written in waveform (.wav) are obtained by scanning existing sounds previously recorded or the capture sounds directly to your computer through special devices, for example, a microphone.

According to Lindstron (1996) for the audio signal scanning, changes its electrical values are converted to numeric representation by a device called an analog to digital converter (ADC – analog to digital convert) – which is a firmware. This numerical representation of the audio wave form can be converted back to an analog signal by a digital to analog converter (DAC – Digital to Analog Converter) for playback. (Figure 4).

During the audio scanning the ADC digitizes both audio signal dimensions (voltage and time). “The scale of time is converted to digital by a process known as sampling (sampling)” (LUTHER, 1995) (Figure 2). To carry out the sampling, the ADC does instantaneous readings of the voltage at evenly spaced time. The sampling frequency or rate is the number of samples read per second. ”

Sinal de áudio - amostragem

Figure 2 – Sampling

The output of this process is the sample flow corresponding to audio signal position at the time of sampling. According to Luther “we can imagine the samples as snapshots, because each is the exact picture of the voltage of the analog audio signal that existed at the time of sampling. At this moment the samples are converted into digital values in a process called quantization (Chart 3). The voltage range is divided by a number of levels or bandwidth – the exact number of levels depends on the number of bits determined for each sample. For example, if 8 bits are used to represent each sample, then the scale 256 will have levels (256 = 28).

Sinal de áudio - quantizing

Chart 3 – Quantizing

For example, the output audio signal of the scanning process shown in Figure 1, in a 4-bit Bandwidth – 16 levels (16 = 24) – with the sampling rate seen in Figure 3, would be the following flow bit: 0011011010101101111011011001010100100010010010001100111011101011.

Still Picture Files in Bitmap Format

This file format is structured in the form of a bitmap, where it transforms the image into a kind of dot matrix; The intersection formed by the crossing of the lines with the columns is called pixel-picture elements-: on a standard VGA video monitor there are 640 columns per 480 lines making a total of 307,200 pixels. This is a highly structured file that contains information regarding type, size and colors, as well as elements of a picture or pixels. Bitmap files can store photo-quality images of all colors; At the moment of reproducing this file the computer mounts it by composing the dots “pixels” side by side, mounting the image as if it were a kind of mosaic.


Formação da imagem - pixel ampliado

Figure 2 – Image Formation

The scanner, digital camera or digital camcorder scans the image, from left to right and from the first to the last line, reflecting the colors or grayscale pixel in electrical values, which are converted into a set of bits that correspond to a color or gray tone in a table. These bits form a kind of map of the pixels of Fig.

digitalização de imagem

Figure 3 – scanning process of an image: figure, and scan result.

For example, to scan Figure 3, a 1-bit color depth – two colors – and with a resolution of 8 lines x 10 columns, obtaining the pixel during the scanning process would be required. The bit stream: 000000000100000000 10000000110000000100000000100000001100000001000000001000000000 which describes pixel is red (1) or white (0).

Videos Scanning

When we watch a video, in fact, we are seeing a succession of still images and slightly different from the previous. This process occurs at a minimum rate, thanks to an optical phenomenon called retinal persistence (Pfromm NETTO, 1998), that causes in our eyes the illusion of movement. The process of scanning videos then resembles the process of scanning each of these still images, plus the scanning of the sound corresponding to the moment of its permanence.

Case Study

Traditionally, hotels often use the video to train their employees. However, when scanned and inserted as a resource in interactive training programs, video can acquire a peculiar syntax and amplify their potential.

The CD-ROM “Front Desk – Addressing the guest at the front desk” is an example. In this training software for hotel receptionists, the video was not simply added to the body of work, to be seen as if it were running on a VCR. Instead, on this CD-ROM the trainess can actually interact with it. He/She can view the sequence they want depending on their needs and decisions.

Training videos demonstrate procedures and attitudes. Receiving the guest at check-in or serving them at check-out; how to perform a wake-up or  a fax service for the customer; how to answer a phone call or how to act as a concierge are examples of videos available on this CD-ROM.

We know, however, that certain procedures require different decisions depending on each situation. If, in our case, an attendant is running a check in with reservation procedures are not the same as a check-in without reservation. If the guest asks to be awakened, their behavior is not fully predictable, as they can readily answer the phone; or they can not hear the ringing of the telephone and stay asleep; they can answer it, but then go back to sleep etc. Each guest behavior requires a different procedure from the receptionist. Obviously it would be impossible to foresee all possibilities. But the most common have been previously recorded and digitized to be studied. Plus, the program readily provides the trainee with a sequence that is a service to interact according to a particular training need.

For example, when making an extension of stay for a guest, the attendant has at least two situations: it can be individual or tourism agency. Each case has its own procedure. When interacting as the program, the trainee should have available the two video sequences: one for each situation. However, the interagent does not need to see the sequence showing the procedure of extended stay for agency guests if he wants, for example, to question the procedure for individual guests. Conventional video does not allow this possibility so easily. It is necessary to rewind or advance the tape to the desired point and this usually takes time. But on the CD-ROM, he can see each of them promptly. The problem, however, is how to organize the sequences in a way that the interacting agent has a map of all the sequences to be seen at will, without having to search for them.

After trying several options to solve this problem we borrowed an old feature that shows visual mode procedures: the flowchart. A flowchart has a problem: he has symbols that may not be known by the person interacting with it: rectangle, ellipse, rhombus. To simplify, we eliminated part of these symbols using only the rectangular shape and transformed all buttons without eliminating, however, their connecting channels. In addition to making the most intuitive flowchart, we use the texts related to each rectangle as titles sequences.

As the flowchart has a hierarchical structure, it is not necessary that the titles completely describe the sequence will be demonstrated. Sometimes the title of the previous button already expressed some of what behavior will be held the following button. For example, at check-out there is the “Pay” button. By clicking this button the interacting see a video sequence showing how to behave in relation to a payment by the guest request. Below this button, there are three: “By card”, “By money” and “By check.” It is not necessary that these buttons clearly define the word “Payment by credit card” or “Pay with cash” because the previous button already makes this clear.

In principle the interactor will see the flowchart as a whole. Sometimes, being too long, the flow chart could not be seen through the computer screen. In this case it was necessary to use scroll bars. Clicking any Flowchart button, it hides, and interacting see the sequence of the corresponding video, for example, sequences to suit a guest: “Good morning, what you want” etc. This video is displayed along with a button to stop and one with double face: “Play / Pause”. When you finish this sequence, you can see it again or choose another way, because the program automatically displays at the end of the sequence decision buttons.

These buttons have different configurations. They can simply present the design so that the user can move forward in the flow chart, ie, the next sequence or return to the previous sequence. If the first rectangle flowchart obviously there will be a button to return. The same can be said for the last rectangle. In this case, there would be a button to move forward. However, there are cases where when moving forward, there is a need to take a decision. For example, a video has the following sequence: “Good morning. I would like to extend my stay,” the guest said. “Certainly, ma’am,” replied the clerk. “Are you an agency client or individual?”. When you finish this sequence, in reality, there are two possible paths in the procedure. So the flow chart will display three buttons. Besides the button to return, there should be two more, prompting the interacting / training which the sequence that he wants to see: that corresponds to the procedure for “guest agency” or “individual guest.” When a situation like this happens, the interactor can choose one of two sequences and will follow in the way he wants. In addition to the function of providing video, Flowchart rectangles also guide the one interacting on the paths already travelled: the rectangle changes color when the user triggers the sequence.

exemplo de fluxograma

Screen 1 – Flowchart with subprocedures

To complete, in this program we also developeded a graphical environment, called “radar”, which shows what were the sequences that the trainee has already interacted with. This radar is an icon that when clicked allows access to the flowchart. Its design is a miniature of it. Like the flowchart, the rectangles change color guiding the interactor on the paths taken.


Screen 2 – Video Preview

Although it was interesting that the person interacting saw the video in a structured sequence from the first button – the one on the top of the flow chart – in fact, the user can subvert it. For example, he may want to see only the sequence that shows thanks; generally in the last flowchart button.

Of course some procedures do not require the receptionist to make decisions, so they are linear. Still, using the flowchart has shown to be a good solution, since video can be viewed from a specific point.


Although visual interfaces have various graphic elements for organization of information, they are not always enough. The graphic expression has once again offered resources for organizing thoughts or complex realities. The flow chart was the solution that we found in visual computer interfaces.

It proved to be quite effective in managing non-linear films because past the learning stage of how to deal with it, the interactor could select the sequence that best suited their needs. Finally, this interactor did not know the flowchart is now also user of this tool for expression of knowledge or graphic procedures.


Prof .: José Eduardo Stefanelli
Prof. Ms. IFSP – Federal Institute of Education, Science and Technology São Paulo.
Research Center Researcher of New Communication Technologies Applied to Education – The School of the Future at the University of São Paulo.
[email protected]

Eduardo Stefanelli

Engenheiro por profissão, professor por vocação