Frequently Asked Questions about eXtremeDB

Summary

This is a list of Frequently Asked Questions about McObject's family of products: eXtremeDB, eXtremeWS and Perst. If you are seeking answers that are not included in this FAQ, please send e-mail to info@mcobject.com. This FAQ is intended to be a first resource for McObject users, developers, and anyone that is interested in our products. It is divided into three sections: General, Technical and Licensing.

List of questions

A. General product-related questions

What is eXtremeDB?
What are eXtremeDB Editions?
What is eXtremeSQL?
What is eXtremeWS?
What is Perst?
How can I evaluate eXtremeDB?
How are software updates handled?
Is source code available for eXtremDB?
Is it possible to get pre-built binary versions of the eXtremeDB runtime?
Who is using eXtremeDB?

B. Technical questions

What performance can I expect from eXtremeDB?
What performance can I expect from eXtremeSQL?
What is the largest database that eXtremeDB can manage?
What is the difference between the "standard" and "shared memory" versions of eXtremeDB?
Does eXtremeDB support transactions?
Does eXtremeDB support multi-threaded concurrent access (simultaneous readers and writers)?
How are process crashes handled? How does the runtime handle database "locks"?
Does eXtremeDB support SQL, ODBC, and/or JDBC?
Are duplicate/unique/compound/descending keys (indexes) supported?
I am confused with the cursor behavior; could you explain the navigation rules?
Does eXtremeDB have a data definition language?
Does eXtremeDB support complex data structures?
What is the memory footprint of eXtremeDB?
How much overhead does eXtremeDB introduce for its own metadata?
What options for persistence are offered by eXtremeDB?
Under what circumstances does the eXtremeDB runtime call malloc?
How difficult is it to port eXtremeDB to different operating systems?
What compilers are used to build eXtremeDB?
Can eXtremeDB operate without an operating system?
Can eXtremeDB operate without a file system?
What capabilities are present in eXtremeDB for application/database debugging?
Does the eXtremeDB XML implementation follow any standards?
Why should I use XML, could you give me a short overview?
Does eXtremeDB support XML representation using attributes?
Where do I find more information about XML?

C. Pricing and Licensing questions

How does McObject license its eXtremeDB line of products?
How is eXtremeWS licensed?
What type of technical support is available for eXtremeDB and eXtremeWS?
Where can I obtain pricing information for McObject products?
Perst is free, isn't it?
Does McObject provide technical support for Perst?

A. Product-related questions.

What is eXtremeDB?
eXtremeDB is an in-memory database system ( IMDS ), developed specifically for use in embedded applications such as industrial control and industrial test equipment software, telecommunications equipment software, etc.

What are eXtremeDB Editions?
In addition to the core product (the Standard Edition), the eXtremeDB family of products includes a number of extensions, called eXtremeDB Editions, that are licensed as optional add-ons. Please follow this link to learn about eXtremeDB Editions that are currently available.

What is eXtremeSQL?
eXtremeSQL is a high-performance implementation of the SQL database programming language for the eXtremeDB in-memory database. With eXtremeSQL, McObject targets the real-time enterprise software market by greatly simplifying programming with eXtremeDB for corporate developers using SQL. The new interface strengthens eXtremeDB's appeal for application development in fields such as banking and securities trading, where real-time responsiveness is a must and SQL is the dominant database language.

What is eXtremeWS?
eXtremeWS, is a small footprint, highly configurable embedded web server for intelligent connected devices. eXtremeWS was designed specifically for the needs of embedded devices. It implements a rich set of capabilities that are essential for embedded development and can be tightly integrated with eXtremeDB, providing a unique environment for implementing a complete data management solution for intelligent connected devices.

What is Perst?
Perst is a high-performance embedded object-oriented database for Java and C-Sharp. Perst is its tightly integrated with the programming language. There is no gap between the database and application data: Perst stores data directly in language objects. Therefore, there is no need for packing and unpacking the code, traditionally required by relational databases and object-relational mapping.

Perst is McObject's Open Source product that is available under a dual license.

How can I evaluate eXtremeDB?
The eXtremeDB runtime is platform-independent, so you don't need the actual hardware platform to evaluate eXtremeDB features and its use, or to build test benchmarks or a prototype for the eventual application. Download one of the evaluation packages from Evaluation Software. Evaluation packages are currently available for all 32-bit Windows, Windows CE, QNX, Linux, VxWorks, INTEGRITY RTOS, Quadros RTXC, LynxOS, Nucleus, HP-UX, and Solaris. These packages include a complete set of the documentation, the eXtremeDB runtime, the schema compiler, and plenty of samples that illustrate various features and programming techniques.

Install the software using the provided installation program. Build the samples according to the readme file. Spend some time reading through the User's Guide and, as different features are discussed, refer to the appropriate samples. Build some tests to measure the performance and memory usage. Contact us if you need assistance with the evaluation process. We will make sure that you get answers to all your questions in a timely manner, and will gladly assist you with writing benchmark tests.

How are software updates handled?
McObject offers an annual technical support and software maintenance contract. This entitles the developer to unlimited technical support through any channel (telephone, email, etc.) and free updates (version number updates to the right of the decimal point, e.g. from 2.0 to 2.1) and a 50% discount on upgrades (version number updates to the left of the decimal point, e.g. from 2.0 to 3.0). With a support contract, upgrades within 90 days of the initial license purchase are free.

Is source code available for eXtremeDB?
Yes. McObject's philosophy is to make the source code available and affordable for developers.

Is it possible to get pre-built binary versions of the eXtremeDB runtime?
McObject provides pre-built binaries of the eXtremeDB library, eXtremeDB extensions and Data Definition Language compiler for some architecture/compiler combinations.

Who is using eXtremeDB?
eXtremeDB provides a fast, cost-effective embedded in-memory database solution in numerous markets,  including consumer electronics, network infrastructure and telecom devices, military/aerospace, industrial control and real-time financial applications. McObject solutions are used by many companies around the globe. A partial list of customers includes:  

Industrial Control
DaimlerChrysler
Tyco Thermal Control
RadiCo
MPDV Microlab GmbH

Consumer Electronics
Seiko Epson
JVC
Pioneer
Genesis Microchip
PEIKER acustic

Network infrastructure
F5 Networks
Seabridge Networks
Hutchinson Telecom
EF Johnson
Motorola

Milltary/Aero
Boeing
EADS
Lockheed Martin
SAIC

Financial
Dalian Commodity Exchange
Stockgroup Media
Breakwater Trading

B. Technical questions

What performance can I expect from eXtremeDB?
The performance of the eXtremeDB runtime depends on many variables such as the processor and memory used, data access pattern, etc. In general, as an in-memory database, eXtremeDB operates many times faster than a conventional on-disk database management system, even if the disk-based DBMS is used with a RAM disk (this is discussed in detail in the white paper Main Memory vs. RAM-disk Databases: A Linux-based Benchmark, available for download at White Papers).

McObject's policy is to assist our customers with the evaluation and benchmarking of eXtremeDB on the platform of interest. We are always ready to construct a reasonable test for your particular environment and application.

What performance can I expect from eXtremeSQL?
Similar to the eXtremeDB core runtime, the performance of the SQL module varies from one environment to another.

What is the largest database that eXtremeDB can manage?
eXtremeDB can manage 2 billion objects per database, with a user-configurable number of database connections (the default is 64). Note that a 32-bit operating environment is constrained by the amount of memory that can be addressed in a 32-bit pointer, usually 2GB or 3GB. So the practical database size limitation is actually a factor of the operating system, not eXtremeDB.

64-bit eXtremeDB on a 64-bit OS overcomes the 2- or 3-GB limitation.

What is the difference between the "standard" and "shared memory" versions of eXtremeDB ?
The Standard version of eXtremeDB holds the database in conventional (local) memory. This is suitable for operating systems with a single-process, multi-task architecture (e.g. VxWorks 5.x) or where the application's requirements do not include concurrent access from multiple processes. The Shared Memory version of eXtremeDB places the database in shared memory which is mapped to the local address space of each process. This approach is better suited to multi-process architectures like Linux, Unix and Windows.

Does eXtremeDB support transactions?
Yes. eXtremeDB transactions support the ACID properties (ACID refers to the principles of Atomicity, Consistency, Isolation and Durability). As an in-memory database, eXtremeDB offers durability in the sense that transactions are durable so long as the database is not discarded.

Does eXtremeDB support multi-threaded concurrent access (simultaneous readers and writers)?
Yes. eXtremeDB is fully re-entrant and thread-safe. The transaction manager enforces concurrent access, and the application need not do anything other than use eXtremeDB' s programming interface for starting, committing and aborting transactions.

I have the situation where if a process crashes, or gets killed in the middle of a transaction (specifically, a write transaction), the lock is still held after the process dies, and all other processes attempting to start a transaction will go to 100% CPU in a busy wait for the lock. Are there any build flags, code alternatives, or any other alternatives or design patterns that you recommend to alleviate the situation?

In version 2.3 of eXtremeDB, there is no recovery from these events; the defect or the signal that caused the abnormal termination of the process with the open transaction could leave the database locked. eXtremeDB locks are based on the simple user-mode spinlock and don't contain any information about process-related data that would allow the runtime to drop the lock.

The process that died while holding the locks will have left the database and its internal structures in an inconsistent state (e.g. certain data updated, but not the indexes, etc). Furthermore, in version 2.3, some of the information needed to rollback the transaction is placed on the application's stack (and is lost when the process terminates)

eXtremeDB version 3.0 provides a facility to abort the transaction of a terminated process by moving the information needed for rollback from the application's stack to the database memory arena and implementing a registry' containing process-related data about current locks.

With respect to releasing the locks (but not to database recovery) in version 2.3, an alternative would be to build eXtremeDB with locks implemented with native OS semaphores System V semaphores, Windows native semaphores, etc.

Does eXtremeDB support SQL, ODBC, and/or JDBC?
eXtremeDB creates a unique programming interface for each database, derived from the programmer's data description for the database. This approach offers many advantages in areas including programming ease, application maintainability, determinism (predictability) and even learning curve. These benefits are described in a white paper titled SQL or Navigational Database APIs: Which Best Fits Embedded Systems? available for download from White Papers .

In addition to the native eXtremeDB programming interface, McObject offers eXtremeSQL , an optional add-on SQL programming interface that implements much of the SQL-89 standard for SELECT, INSERT, UPDATE and DELETE statements. eXtremeSQL also implements many extensions to support eXtremeDB - specific features. There is not yet an ODBC or JDBC driver for eXtremeSQL .

Are duplicate/unique/compound/descending keys (indexes) supported?
Yes. eXtremeDB supports all of the above index types, as well as hash-based indexes, user-defined Object Identifiers (OIDs) and system-generated Auto-Ids (similar to sequence or serial data types in other DBMSs).

I am confused with the cursor behavior; could you explain the navigation rules?
An eXtremeDB tree index is a tree-like structure in the application's memory that contains a collection of keys. A key can be a simple one (based on a single field) or a compound one that contains more than one field. The tree-index stores the keys in sorted order. In order to support sorting, the tree-index must have a compare function. Regardless of the key type (simple or compound), the compare function must be able to compare two keys and determine whether the first key is less (compare returns -1), equal (0) or greater (1) than the second key. The tree-index algorithm sorts the keys based on this comparison. It is important to note that the tree-index algorithm sorts the keys based on their relative weight, determined from the compare function results, rather than on the key's value . Download the rules and examples here.

Does eXtremeDB have a data definition language?
Yes. eXtremeDB has a database definition language (DDL) and a database dictionary. The DDL looks very much like C++. It is created with a text editor and processed by our schema processorwhich generates the .h and .c files that contain the programming interface for the databaseand is compiled and linked with the rest of the application code.

Does eXtremeDB support complex data structures?
Yes. There is no theoretical limit to the complexity of the data you can define with eXtremeDB 's DDL. eXtremeDB supports C/C++ structs, nested structs, fixed and variable length arrays of any supported data type, and BLOBs.

What is the memory footprint of eXtremeDB?
eXtremeDB's footprint varies by CPU and compiler, ranging from 45K to 130K. eXtremeDB extensions add to the footprint slightly. eXtremeSQL adds approximately 100K.

How much overhead does eXtremeDB introduce for its own metadata?
The overhead imposed by eXtremeDB core runtime varies based on the application data layout and number of indexes defined by the database designer. In most cases eXtremeDB will add about 20% to the size of the raw data for its own pointers and other structures it requires to maintain indexes and other references to the data.

What options for persistence are offered by eXtremeDB?
eXtremeDB offers a number of persistence options. First of all, the core eXtremeDB has functions to write a database image to a stream such as a socket, a file or a pipe, and to load a database from a previously saved image. These operations run within the context of a transaction, assuring that a transaction-consistent image of the database is written (or read).

Second, eXtremeDB Transaction Logging Edition runtime captures database changes and writes them to a file known as a transaction log. In the event of a hardware or software failure, the eXtremeDB runtime can recover the database using the log. Logging is performed through periodic checkpoints, where the image of the database is saved to persistent storage, and all intermediate changes to the database are written to the log files.

Third, eXtremeDB High Availability Edition maintains multiple identical database instances within separate address spaces. Typical hardware configurations include:

• Multiple processes or threads within the same hardware instance
• Two or more boards in a chassis with a high-speed bus for communication
• Separate address spaces on boards connected via industry standard communication media and protocol such as RS-485/RS-232, Control Area Network (CAN) or Ethernet
• Two or more controllers connected via a proprietary communication media and/or protocol
• Separate computers on a LAN

Finally, eXtremeDB 3.0 provides support for the databases created in the non-volatile memory, NVRAM. The runtime recovers in-memory database after failure, provided that the in-memory image is in a battery-backed SRAM which retain their contents in the absence of system power.

Under what circumstances does the eXtremeDB runtime call malloc?
The eXtremeDB core runtime does not call malloc. Instead, eXtremeDB implements its own, highly optimized memory managers.

How difficult is it to port eXtremeDB to different operating systems?
eXtremeDB has been designed and implemented to be highly portable. Currently, eXtremeDB runs on all Win32 platforms, many Unix platforms, and various embedded operating systems including VxWorks, INTEGRITY and QNX. The list of supported platforms is constantly expanding.

What compilers are used to build eXtremeDB?
The eXtremeDB runtime can be built with the Microsoft Visual C/C++ compiler, gcc and all ANSI C compliant compilers. The eXtremeDB Data Definition Language compiler is currently built with Microsoft Visual C/C++ 6.0 for Windows and g++3 for Unix/Linux platforms.

Can eXtremeDB operate without an operating system?
Yes, in single-threaded mode. In this mode the eXtremeDB runtime does not use any operating systems services, or the C runtime library.

Can eXtremeDB operate without a file system?
eXtremeDB is an in-memory database system and does not require any file system to operate. If a file system is present, to allow data persistence through a system crash, eXtremeDB provides backup functionality via its stream-oriented save and load interfaces. eXtremeDB can backup databases not only to a file, but also to any stream device that is available in the operating environment, such as a pipe or a socket.

What capabilities are present in eXtremeDB for application/database debugging?
eXtremeDB is distributed in two versions: debug and release. In debug mode, eXtremeDB provides excellent debugging capabilities for developers. There are a number of traps throughout the eXtremeDB code that facilitate detection of application errors. For example, if an incorrect pointer is passed into eXtremeDB , chances are that a fatal exception will be raised by the eXtremeDB library before the pointer is used, which makes it possible to examine the call stack and find the source of the corruption. The compile-time type checking of the C compiler applies to the methods that eXtremeDB uses to access stored data. Methods that are generated to provide access to a certain object type always take a reference to that data type as a parameter. If a mistake is made, a compiler error will be generated. In release mode, all the above traps are removed from the eXtremeDB runtime code, and many functions inside the runtime are inlined. That can significantly increase the performance of the eXtremeDB -based application.

Does the eXtremeDB XML implementation follow any standards?
The eXtremeDB XML interface is developed in accordance with the W3C SOAP encoding recommendations. These recommendations are found on the W3C web site :

SOAP Version 1.2 Part 0: Primer
SOAP Version 1.2 Part 1: Messaging Framework
SOAP Version 1.2 Part 2: Adjuncts

eXtremeDB XML schema encoding is developed in accordance with the W3C standards described in the following documents:

XML Schema Part 0: Primer Second Edition
XML Schema Part 1: Structures Second Edition

Why should I use XML, could you give me a short overview?
XML is a clearly defined way to structure, describe, and interchange data. Data in this context means every conceivable kind of data, including all forms of eXtremeDB supported complex data. Therefore, XML is an ideal medium that allows exporting and importing real-life data from and to eXtremeDB databases.

To learn more about XML visit the following link the W3C's Extensible Markup Language page.

Does eXtremeDB support XML representation using attributes?
Yes, eXtremeDB XML supports the alternative representation with attributes. Attributes are used to provide additional information about elements. In essence XML elements can have attributes in the start tags, just like HTML. Please see W3C specifications for details. Data can be stored in child elements or in attributes. There are no rules about when to use attributes, and when to use child elements. However, according to SOAP recommendations XML should try to avoid using attributes.

Where do I find more information about XML?
Online, there's the XML Specification and ancillary documentation available from the W3C.

C. Pricing and Licensing questions

How does McObject license its eXtremeDB line of products?
McObject issues two types of licenses for eXtremeDB: development licenses and deployment licenses. McObject licenses developers on a per-platform, per-developer basis. eXtremeDB deployment licenses are negotiated on a case-by-case basis due to the wide variety of business models and devices/applications that employ our technology.

How is eXtremeWS licensed?
The web server product is licensed with source code on a per-developer, per-platform, per-application basis, and is royalty free.

What type of technical support is available for eXtremeDB and eXtremeWS?
Annual support contracts are 15% of the total licensing fee, and entitle developers to unlimited telephone technical support, minor updates (release number increments to the right of the decimal point), and a 50% discount on major upgrades (release number increments to the left of the decimal point) during the support period. Major upgrades are free if issued within 90 days from the start of the inaugural support period. An annual support contract is mandatory for the first year.

Where can I obtain pricing information for McObject products?
Please contact us via e-mail at sales@mcoject.com or call our office.

Perst is free, isn't it?
No, Perst is not free. McObject Perst is available under a dual license. Under the GPL, you may evaluate the source code free of charge and you may use Perst free of charge in an application for which the source code is also freely available. If you wish to use a Perst in an application but do not or cannot redistribute your application source code, you can use Perst under a commercial license. 

Does McObject provide technical support for Perst?
Yes. Annual support and software maintenance is offered for Perst, for $2,000 per developer per year. In addition to email and telephone support, and with a commercial license, this entitles you to distribute the most up-to-date version of the database run-time code with your application. Without a Support and Maintenance Agreement you may distribute the software version that is current as of the date of the commercial license between you and McObject LLC, or the expiration date of your last Support and Maintenance Agreement.