Summary of Functions

This document is a more or less complete list of functions as of August 9, 2003.  Parts of this document will overlap with the help document for the webpages.

This document is a brief overview of the Lifestyle-Integrated Automation Machine (LIAM). A conceptual view of the system is depicted in the diagram above.

The system is essentially a comprehensive home automation system integrated with a task management system. It is controlled through a variety of user interfaces (e.g., "LifeConsole") and records all key "happenings" in a database ("LifeChronicle").

The home automation system can be categorized into functions related to occupancy/security (which in turn controls many other functions), general household functions (e.g., aquarium, spa, heating, lighting, etc.), desktop computer functions, audio/video functions, telephony functions (e.g., voicemail, phone transfer) and miscellaneous.

The task management system can divided into functions for planning of tasks and the execution of tasks. Most of the functionality is provided by Outlook with several programmatic extensions.

• Occupancy is based on use of phone, use of LCD keypad, asleep mode, hallway IR, foyer IR, or any exterior door. Each device is assigned a "certainty". Certainty scoring important for lights vs. alarms! 
• Occupancy "decay" is also provided. This means that the certainty of occupancy diminishes over time. For example, if a person hasn’t been detected for 10 minutes, certainty goes from "Occupied" to "Probably Occupied". 
• If entry occurs into the house without the keychain remote, this "non-validated" entry causes a silent alarm including paging.
• The garage entry/exit sequence begins with the use of a keychain remote. Part of the sequence is automatically closing/opening the garage door by pulling car out/in. Doing so sets occupancy appropriately.
• A keychain remote may also open the front door. Alternatively, a LCD keypad button can open the door.
• Once unoccupied, the system turns off computer display, lights, heat, A/V, etc.
• Any door that is left ajar is indicated on the LCD keypads with a flashing light. After 40 minutes, a warning is spoken throughout the house.
• If the garage door is manually opened, it will automatically close (40 minutes).
• If someone approaches the front door, this is announced throughout the house, all TVs show the front camera and the accompanying sound, and the camera scene is recorded on DVR. 
• The ringing of the doorbell is played throughout the house and whether the door was answered or not is logged.

 General Household Functions

• Spoken summary of key events when entering house or as requested. Events include status of door closures, phone calls placed & received, email and voice messages, occupancy events, temperature, etc. Short and long versions are available.
• Guest and vacation modes can be set, modifying the behavior of some systems. For example, when guest mode set, setting goodnite will not turn off all the lights.
• Vacation mode is automatically set if house not occupied for 24 hours.
• Time reminders are spoken when it’s time to go to bed or time to go to work, or it’s a pleasant temperature outside (important for cold San Francisco!).
• Time is announced on the hour throughout the house when awake and on request (quietly) late at night.
• Two reminder timers can be set. The first timer is a countdown of minutes and the other one is set with a specific time. In the case of the latter, if house is not occupied when alarm time comes, owner is automatically paged. 
• Fresh water is automatically added to saltwater aquarium to maintain optimum salinity levels.  
• Whole house lighting scenes are selectable or change automatically throughout the night. Lights also are adjusted automatically when viewing a video player (i.e., DVR, VCR, DVD).
• Path from bedroom to kitchen is lit automatically or by request if house is in asleep mode and someone gets out of bed.
• One of four HVAC setpoints are set depending on whether the house is occupied, unoccupied, asleep, or recently awakened states. Whole house fan activated on exceptionally hot days.  Temp may be adjusted from any LCD.
• Comprehensive goodnite sequence programmed: turns off A/V, dims down and off lights over 10 minutes, adjusts HVAC, sets house alarm, closes bedroom shade, turns on electric blanket (if needed), turns off bedroom fridge for 1 hour (for quiet), turns on ambient sound, sets wakeup alarm for 7:40 hence (unless overridden). 
• Comprehensive wakeup sequence activated by wakeup alarm: lights dim up over 13 minutes and then TV comes on. Once is owner is up and about, TV goes off (unless overridden), bedroom, living room, & dining room shades opens, espresso maker goes on, several lights adjusted and email inbox displayed on main computer. 30 minutes later, espresso maker goes off and several lights are adjusted. 
• Multiple zones of sprinklers are on automatic timer.
• Spa is automatically heated according to a schedule. (Disabled during vacation mode.)  A service function to shut off jet after 10 minutes available for when adding chemicals.

 Desktop Computer Functions

• Ability to view any computer on any TV. Each major room has an IR keyboard as well.
• In addition to displaying a computer on a TV, one of several programs can also be launched. Once a program is launched, any number of keystrokes automatically issued to that program as needed. For example, with one button the Yahoo Weather page can be launched and then a page down automatically sent to the browser to scroll down to the forecast. Programs launched currently include LIAM website pages, to-do list, email client, weather webpage for forecast, finances program to calculate and display net worth (requires password), and several picture galleries. 
• The program(s) are automatically exited if all TVs cease to show that computer.
• Automatic computer "housekeeping" functions are also provided: minimize open programs on all computers at the end of day, when unoccupied, etc. and display email inbox at the beginning of day. 
• Any email that arrives where the sender is in the Outlook address book is announced throughout the house. 
• Computer display turned on/off according to occupancy or asleep modes.
• A computer process ensures all computers (including automation controllers) have the same time. This is checked several times a day.

 Audio/Video Functions

• The following audio sources are selectable: environmental sounds (from a dedicated CD player), one of 101 CDs, FM radio, satellite music, cassette, or turntable.
• For radio and satellite music, one from a list of specific stations can be selected.
• For CD, one of 10 playlists can be selected and a CD within that playlist is randomly played. Also, random (not from a playlist) and highlight play can be selected. The current playlist and CD is made available on the Internet (see below).
• CD with "ambient" DVD video can be selected. This DVD is selected from an assigned position in a multi-disc player. 
• Any audio zone can be turned on or off.
• The following video sources are selectable: DVR (Tivo), DVD player, VCR player, computer display or front door camera. Surround sound selected as appropriate.
• A black-out shade lowered in bedroom and lights adjusted if a TV player (i.e., DVR, DVD or VCR) is selected in the bedroom and it's dark outside.  Likewise, dimming shades lowered in living room & dining room if a TV player is selected in the living room and it's dark outside.
• Volume automatically adjusted according to typical loudness of source component. 
• Volume automatically muted when phone rings or in use or when the system is making a whole-house announcement.
• A small video monitor (without sound) automatically switches on at dark to show the current DVR channel. It may also be manually controlled.
• Convenience controls provided on all LCD keypads depending on audio / video source selected for primary controls such as volume, channel, etc.

 Local Phone Functions

• Phone ring plays throughout house. Caller ID sent to all LCD keypads. If the caller ID is found on the personal list, the person's name is announced throughout the house. The DVR is also paused.
• Use of the phone will mute any A/V system sound and will pause the DVR if currently playing. 
• System will whole-house announce the phone being in use at 40, 80, 120 and 160 minutes (in case it's off the hook).
• While a call is in progress, the following functions are available: 1) ability to get current call time, 2) record a message, 3) place phone on hold, 4) put call on whole-house audio system, 5) open front door.
• A "local" phone mode is available with the following functions: 1) leave a local message (memo function), 2) get messages (local & standard), 4) phone to phone intercom, 5) whole-house paging.
• Pressing "*" in any phone mode will provide the user with help.

 Outside Phone Functions

• If phone not answered, caller is directed to voicemail unless on the personal list. Being on personal list is determined by caller ID, or if blocked, by the entry of a unique code. If on the personal list, the caller is greeted by name and either transferred to a remote phone automatically (unless disabled) or informed that the owner is asleep (if applicable) or routed to voicemail. Callers not on the personal list are routed to voicemail. 
• The remote phone is automatically set to a cell phone unless overriden while at home or remotely.
• If the caller is the owner, a phone menu is spoken to 1) retrieve status, 2) leave a message, 3) listen to messages, 4) turn on vacation mode, HVAC or the spa, 5) change remote phone, 6) whole-house announce, 7) listen in to the house.
• Pressing "*" in any phone mode will provide the user with help.

 Miscellaneous Functions

• Inside and outside temperature logged every hour (has been since 1998!).

Task Management

• System maintains a database of tasks from which to choose from, organized by category and importance (e.g., clean garage). This list is normally reviewed weekly. This is referred to as the Chooser list. The system also maintains a database of dated items (including recurring), which are automatically chosen (e.g., fertilize plants). Items are placed into the weekly view.
• Items in this Chooser List can be "sticky" or not. Sticky items are put back into the list once marked complete vs. non-sticky which when completed do not go back to the list.
• Once items are put into the weekly view, they are assigned a day of the week (DOW).

• A day's tasks are managed near realtime. As items are handled, they can be deferred to another day of the week, put back into the Chooser list or marked complete. Completed tasks are logged and time stamped with date and time.
• For convenience, items may be "chunked" (e.g., all outdoor tasks) together.
• To do items can be sourced from several points: by typing the task on the computer, by entering it in a PDA, by recording a voice memo on the PDA (a task is automatically added to transcribe the recording), by leaving a phone message locally or by call in, or by email using a special subject line including from a cell phone. [No way to forget that item!] 
• Additional functions are provided for PDAs to ensure functional parity between a desktop computer and the PDA.

• An "expression" is any non-actionable item created by the system owner.  It may be text or pictures.  It is often used as a diary.
• For text, an expression may be created by using the Outlook note feature, writing up a text file and giving it a filename of a certain form, or sending email with a special subject line.  Keywords may be added to the text within braces ("{}")
• For pictures, an expression may be created by giving any picture (JPG or GIF) a filename of a certain form.  Keywords may be added by placing them in braces ("{}") in the filename.
• Expressions with the keyword "public" will automatically appear on the LIAM website bulletin board.

• All file changes (e.g., new files) in the My Documents folder or desktop are logged. 
• The Outlook address book is automatically extracted to a file every couple of days for caller ID, email matching, etc. by other parts of the system.

LifeChronicle is a database, arranged in chronological sequence, of things that happen throughout the day in the life of the system's owner. It captures this information completely automatically but it may be annotated. On an average day, the Chronicle can easily contain 100 individual events. These are selected from thousands of smaller events such as walking down a hallway.

LifeChronicle is feed by both the Home Automation and Task Management systems and consists of:

• a record of all incoming and outgoing phone calls including names and phone numbers when possible,
• a user’s activation of most automation functions (e.g., turning on of a DVD player, adjustment of the light scene, etc.),
• all incoming email where the user is in an address book,
• all visitors to the front door,
• all occupancy events,
• all completed tasks from the task management system,
• all "expressions" including text and pictures,
• the name of all modified files in My Documents or the Windows desktop
• the weather is recorded every hour

The resulting record is an objective chronicle of the owner's life. The concept for this comes from many sources including a technology used by large businesses today called "data warehousing" whereby all external events (e.g., customer contacts) are recorded for subsequent use.