AITEC Contract Research Projects in FY1997 : Proposal

(22) GUIdeLA: A Constraint Language and System for Constructing Graphical User Interfaces

Dr. Neng-Fa Zhou, Associate Professor,
Faculty of Computer Science and Systems Engineering,Kyusyu Institute of Technology



[Contents]

  1. Background of the research
  2. Purpose of the research
  3. Contents of the research
  4. Project Organization/Research Method
  5. Resulting Software

[Background of the research]

Building a user-friendly Graphical User Interface (GUI) by using a language like Visual C++ or Java is difficult and time-consuming. This is because these languages require the users to specify many details concerning the geometric data about the layout and also the sequence in which graphical items are displayed and the sequence in which events are handled. People have noticed that constraints is a useful tool for constructing GUIs. A constraint is a relation between objects that must be maintained. Finding a layout or a sequence of actions that satisfy the constraints is done by the system, instead of being the responsibility of the user. This can greatly reduce the burden of users and dramatically improve the productivity.

Many GUI development systems enable the users specify some kind of constraints, including Intelligent PAD developed at Hokaido University, Garnet and Amulet developed at CMU, ThingLab II and SkyBlue developed at Washington University. Although using these systems makes GUI development much easier than using conventional visual languages, none of these systems has been widely used. The reasons can be summarized as follows. First, these systems do not support nondeterminism. They require a one-to-one mapping between a GUI specification and the final layout. To enforce this, these system either only supports a restricted form of constraints, such as one-way constraints in Amulet, or ask the users to provide enough information, such as preferential constraints in SkyBlue. Second, these systems are all interpreters. Besides the problem of efficiency, interpreter-based GUIs are difficult to be integrated into other applications. To integrate a GUI into an application, one has to integrate the whole interpreter into it. This problem becomes severe when one wants to integrate a GUI into a Java applet because Java applets are usually executed remotely and must be as small as possible.


[Purpose of the research]

This research aims at designing and implementing a new constraint language, called GUIdeLA, for GUI development. GUIdeLA has the following three features besides those nice features in the existing languages:

(1)It is truly declarative. After learning about the basic classes of graphical objects already available, one is able to define easily new classes of graphical objects by specifying attributes, components, constraints, and actions associated with the objects.

(2) GUI specifications are compiled into some intermediate forms that can be easily integrated into other applications.

(3) the mapping from GUI specifications to intermediate forms is one-to-many rather than one-to-one, and the system is able to help the users select satisfactory GUIs. With nondeterminism, the users can sometimes get satisfactory GUIs even without specifying any of the details about the layout.

We choose Java as the intermediate language and B-Prolog
( http://www.cad.mse.kyutech.ac.jp/people/zhou/bprolog.html) as the implementation language for the compiler. Java is chosen because it is rich of graphical libraries and is platform-independent, and mostly importantly, the Java code generated by the compiler can be integrated into Java applets and run over the internet. B-Prolog is chosen because Prolog is an excellent language for writing compilers and the finite-domain constraint compiler provided by B-Prolog is useful for solving constraint satisfaction problems generated from GUI specifications.


[Contents of the research]

The research contents include designing and implementing GUIdeLA. In the following, we describe possible issues we face and the policies we will take to solve these issues.

Design of GUIdeLA

The first issue is how to determine the syntax. We adopt Java-like syntax which is easy for Java and C programmers to learn. A GUIdeLA class of graphical objects has the following syntax:

class <ClassName> [extends <ClassName>] {
  <ComponentsAndAttributes >
  constraints { ... }
  actions { ... }
}

Just like a Java class, a GUIdeLA class can inherit some information from another class. The <ComponentsAndAttributes>declares the components that compose the the objects and the attributes associated with the class. A component is an object of a built-in class or a user-defined custom class. The constraints section specifies the relations among the attributes of the class as well as those of the components. A constraint is either numeric or symbolic. A numeric constraint is a relation among the numeric attributes. GUIdeLA allows the users to specify not only linear constraints but also non-linear constraints. A symbolic constraint specifies how the components relate to each other. For example, the symbolic constraint inside(O1,O2) says that object O1 must be inside object O2. With symbolic constraints, the users can specify many layout constraints without using the attributes explicitly. Theactionssection specifies a list of event-action pairs. When an event has happened, the corresponding action will be take to handle it.
Another issue is to design an easy-to-use API (APplication Interface) for the built-in classes of graphical objects. Java has a rich library of graphical classes. However, one may feel it difficult to use these classes if he/she does not know well the inheritance relationships among the classes.

Implementation of GUIdeLA

A GUI specification in GUIdeLA specifies a Constraint Satisfaction Problem (CSP). The system first extracts the problem from the specification and then invokes the constraint solver to solve the problem. After obtaining a solution, we get an instance of the original GUI specification. The system then translates the instance into a Java program which displays the layout.

If the layout is unsatisfactory, the system will try to find another solution to the CSP and generate another Java program. This process is repeated until a satisfactory layout is found.

A GUI specification is said to be static if no attribute values rely on inputs or the positions of movable objects, and dynamic otherwise. For a static specification, the generated Java program only needs to display the layout. However, for a dynamic specification, the generated Java program must not only display the layout, but also maintain the constraints that have not yet been solved by the compiler. An ideal compiler for dynamic specifications should generate Java programs as small as possible.

There are possibly a large number of solutions to a CSP. To have near-optimal solutions come first, we have to order the candidate values for attributes in a proper order and define some constraints for the built-in classes that can exclude bad-looking layouts as solutions.


[Project Organization/Research Method]

(1) Project Organization

 Name Affiliation
Principal InvestigatorZhou Neng-FaKyushu Institute of Technology
Cooperate ResearcherMcDermott DrewYale University
Cooperate ResearcherHe ShuijiKyushu Institute of Technology
Cooperate ResearcherHonda TakafumiKyushu Institute of Technology
Cooperate ResearcherKaneko SosukeKyushu Institute of Technology


(2) Research Method

Zhou is responsible for designing GUIdeLA and completing most part of the implementation.
McDermott will investigate how AI technologies, such as spatial reasoning and temporal logic,can be applied to this project.
Honda and Kaneko will be responsible for developing the API, some part of the compiler, and writing examples.


[Resulting Software]

(1) The name of the software

GUIdeLA

(2) Functions and features of the software

GUIdeLA is a language and system for constructing GUIs declaratively. To define a class of graphical objects in GUIdeLA, one only needs to declare what components compose the objects and describe how the components relate to each other with constraints. The positions of the objects are computed automatically by the system based on the constraints.

The GUIdeLA system compiles GUI specifications to Java programs. As there may exist different ways to satisfy a specification, the mapping from specifications to Java programs is usually one-to-many. The GUIdeLA system is not only able to help the users select satisfactory layouts, but also able use some heuristics such that good-looking layouts may come out first.

The system treats static and dynamic specifications differently. For static specifications, the generated Java programs display the satisfactory layout. In contrast, the generated Java programs for dynamic specifications are responsible for not only displaying the layout but also maintaining the constraints that rely on some inputs or positions of movable objects.

GUIdeLA has the following two advantages over the previous systems for GUI construction: First, the system supports one-to-many mapping from GUI specifications to Java programs. With nondeterminism, the description level becomes much higher, and the users do not need to provide to the system with all the geometric data about the layout. Second, GUIdeLA is a compiler rather than an interpreter. The Java code generated for the GUI can be modified and be integrated into other applications such as Java applets.

(3) Structure of the software

  • The final system has four components: the syntax analyzer; the translator that extracts CSP from specifications; the compiler that compiles specification instances into Java; and a library of built-in classes of graphical objects.

  • This software will be built from scratch. To run the software, one is required to first install B-Prolog, which is free for non-commercial use.

  • Some ideas of the existing systems such as Amulet developed at CMU will be employed, but no code will be reused from these systems.

(4) Related IFS programs (if any)

The software does not directly use any IFS programs, but has close relationship with the Fifth Generation Project because of the following: The kernel concept underlying GUIdeLA is constraints; the implementation language is Prolog; and final product can be integrated into many of the IFS programs.

(5) Required program language/OS/software packages

The resulting system will run on almost all platforms including UNIX and Windows machines.

(6) Expected size of the software

The final software is estimated to be 10,000 lines in Prolog.

(7) Expected way of use of the software

  • How this software will be used, who will be expected users?
    A user-friendly graphical user interface is so important that no software can lack it. GUIdeLA can be used to develop GUIs to be integrated into other applications such as Java applets. GUIdeLA is easy to use because of its declarativeness. Users who do not even know Java at all can use GUIdeLA to develop some GUIs and include them in their WEB pages. For this reason, GUIdeLA will be of interest to a wide range of users.

  • Advantage of the software from users' point of view.
    The most important advantage of this software to the users is that users can design GUIs without programming the details. This not only enhances the productivity of programmers, but also opens GUI construction to non-programmers.

(8) Documents which will be added to the software

Specification, User manual, etc.


www-admin@icot.or.jp