<< | IBExpert DemoDB | >>

IBExpert benchmarks

Benchmarks are normed testing techniques, used to evaluate and compare the performance of IT systems according to certain predefined criteria. They are a vital tool when the performance of databases and/or hardware needs to be assessed objectively.

Objective testing of dynamic transaction processing performance is now possible using Firebird, IBExpert and PHP together with a simple DVD Store database. With ever-tightening IT budgets, escalating productivity demands and a climate of negotiable pricing, how much new server capacity can a company actually afford? Researching street prices and making sense of apples-to-oranges price/performance statistics when comparing systems can be a daunting and time-consuming task. The absence of benchmarks for websites with dynamic content has been a major impediment to research in this area.

The idea originally comes from the German trade magazine, c’t Magazin, which launched a database contest in Germany in October 2005. An English translation of this article may be found at: http://firebird.sourceforge.net/connect/ct-dbContest.html. HK-Software has decided to offer this benchmark to its Firebird and InterBase® users as a method of testing performance for dynamic web applications where transaction processing is a key feature.

The DVD Store – objective performance comparison

This solution uses 'DVD-Store', a project that the Linux department of the Dell computer company developed in recent years as a test for its internal server laboratory. It is called DS2 and it is free. Modeled solely on a reference implementation of a typical e-commerce application, it offers more than just the usual benchmarks: in addition to providing information about the behavior of a database system in authentic conditions, it also indicates something about its application with middleware and server platforms. It enables users to appreciate the power of a solution relative to the amount of effort needed to achieve it.

The DVD Store Version 2 (DS2) is a complete online e-commerce test application, with a backend database component, a web application layer, and driver programs. The design goals of both the database component and the midtier application were to utilize many advanced database features (transactions, stored procedures, triggers, referential integrity), at the same time ensuring ease of installation and simple construction; to define a set of functional requirements that can be run on any transaction processing system, regardless of hardware or operating system, and still guarantee end-users an apples-to-apples comparison. The DVD Store may be used to test databases or as a stress tool for any purpose. The benchmarks are modeled on actual production applications and environments rather than stand-alone computer tests, which may not evaluate key performance factors such as user interface, communications, disk I/Os, data storage, and backup and recovery.

Its intended use as a laboratory application explains the simple user interface, and the implementations do not use any session management, but carry the session parameters in hidden form fields. However the application comes with several features that make it very suitable for benchmark testing, such as the typical OLTP (On-Line Transaction Processing) features which form part of the essential requirements. DS2 comes with ready-to-use backend implementations not only for Firebird, but also for MySQL, Oracle and MS SQL which offers an interesting insight into the differences and common areas of the database systems.

Requires only open-source components

HK-Software has implemented the DVD Store using popular Open Source software: an Apache™ Web server, the PHP scripting language, and the Firebird relational database, a selection representative of many dynamic content websites online today.

The workload in the DVD Store benchmark is performed in a controlled Internet commerce environment that simulates the activities of a business-oriented transactional web server. It measures the performance of systems supporting users browsing, ordering, and conducting transaction-oriented business activities. The simple DS2 database structure contains merely eight tables, includes however all that is needed to run a small or medium web shop.

The release comprises data generation programs, shell scripts to build data for 10 MB, 1 GB and 100 GB versions of the DVD Store, database build scripts and stored procedures, and PHP web pages and a C# driver program. The full software package can be downloaded from http://www.ibexpert.com/download/ - go to the ds2 directory and save the ds2.zip file to your hard drive. Please refer to the IBEBenchmarks Installation Notes for further information. Questions can be mailed to documentation@ibexpert.com, or posted in the IBExpert forum: http://www.firebirdexperts.com/.

back to top of page

IBEBenchmarks installation notes

Published by HK-Software 2005.

Authors: Debra J. Miles, Andrea Schmidt, Alexander Sowarenko, Manuel Morbitzer.

COPYRIGHT © 2005 HK-Software. All rights reserved.

All IBExpert brand and product names are trademarks or registered trademarks of HK-Software in Germany and other countries. All Borland brand and product names are trademarks or registered trademarks of Borland Software Corporation in the United States and other countries. Firebird is a registered trademark of FirebirdSQL Foundation. Other brand and product names are trademarks or registered trademarks of their respective holders.

Introduction

Objective testing of dynamic transaction processing performance is now possible using Firebird, IBExpert and PHP together with a simple DVD Store database.

For the more experienced developers, who know what they are doing with regard to the installation of the software, and only require a guide to creating the database and data, there is a comprehensive README.TXT in the DS2.ZIP, which is automatically extracted to C:\DS2\. These Installation Notes go into more detail, providing a step-by-step guide for those with perhaps less Windows experience or who are installing the software components for the first time.

This comprehensive Installation Guide can be read online at: http://ibexpert.net/ibe/pmwiki.php?n=Doc.IBEBenchmarks.

There are two ways to run IBEBenchmarks:

  1. Using Apache™ and PHP to create a full web application emulation. Here all four software components need to be installed along with the DVD Store DS2 download.
  2. Using only Firebird and IBExpert, working directly in the database. In this case it is not necessary to install Apache™ or PHP.

The following chapters cover the download and installation of the following five components:

So, let's get started!

Download and install the Apache™ server

The Apache™ HTTP Server is a powerful and flexible HTTP/1.1 compliant web server. Originally designed as a replacement for the NCSA HTTP Server, it has grown to be the most popular web server on the Internet. As a project of the Apache™ Software Foundation, the developers aim to collaboratively develop and maintain a robust, commercial-grade, standards-based server with freely available source code.

The Apache™ server can be downloaded free of charge from: http://httpd.apache.org/download.cgi

The current release can be downloaded from one of the mirrors. Select a mirror and then select the file for your operating system from one of the files listed under: Apache™ HTTP Server 2.0.55 is the best available version; either by clicking on the file name, or right-clicking and saving to the hard drive. We recommend the Win32 Binary (MSI Installer) for Windows: apache_2.0.55-win32-x86-no_ssl.msi. You may need to attempt a couple of mirrors, or even a backup mirror (simply select from the pull-down list) before being able to download successfully. Please note that Apache™ only runs on ports 80 or 8080 (for own use). So you will need to ensure that you have no other web servers using these ports, before installing the Apache™ server.

Now you're ready to install the Apache™ server. An Installation Wizard guides you through the installation:

Progress through the installation until you come to Server Information. If you are working locally, you will need to enter your Computer Name in both fields, Network Domain and Server Name.

To determine your Computer Name in Windows, go to the Windows My Computer and select the right-click menu item System Properties (alternatively use the option Control Panel / System). On the second page you will find the computer name (Full Computer Name). Enter the Adminstrator's email address (note: this field is obligatory, i.e. it has to be filled in, even if you don't need it!), and finally select whether Apache™ should be used on port 80 as a service for all users (recommended option), or whether port 8080 is to be used (only for the current user when started manually).

The next dialog offers two installation types: Typical (recommended) and Custom (only for advanced users). Then all you need to do is to either accept the recommended drive and directory path or alter as wished, and finally press the Install button to proceed with the installation.

back to top of page

Adapting the httpd.conf file

Following installation it is necessary to configure the Apache™ server for PHP. We therefore need to make a number of alterations in the Apache™ httpd.conf file, which can be found under C:\Programs\Apache Group\Apache2\conf. Open this text file, and make the following modifications and amendments (the # symbol indicates a commented row):

Integrate the PHP Parser

Simply insert one of the following at the beginning of the httpd.conf file:

  • load it as a module in Apache™ (recommended option) - integrate the php4apache.dll file for Apache™ 2:
 LoadModule php4_module "c:/php/sapi/php4apache2.dll"
 AddType application/x-httpd-php .php 
  • or as CGI:
    ScriptAlias /php/ "c:/php/"
    AddType application/x-httpd-php .php
    Action application/x-httpd-php "/php/php.exe"

The document directory (for HTML or PHP files) can be altered, for example, to c:/htdocs). To do this it is necessary to modify the following rows:

BeforeAfterAction
DocumentRoot "C:/Programme/Apache Group/Apache2/htdocs"DocumentRoot "C:/htdocs"Alter row
<Directory "C:/Programme/Apache Group/Apache2/htdocs"><Directory "C:/htdocs">Alter row

Simply copy the Before directory paths, search for them in the httpd.conf (Menu Edit / Search or [Ctrl + F]) and replace them with the After paths (Menu Edit / Replace or [Ctrl + H]).

We have now specified where the web server should run, so can save your changes and close the httpd.conf file.

Important!

  1. Don't forget to create a new directory C:\htdocs (or however you have named your document directory in the configuration file) and
  2. you will need to restart the Apache™ server after changing the configuration for the changes to become effective (open the Apache™ Monitor and press Restart or alternatively use Windows Control Panel / Adminstrative Tools / Services / Apache).

Now you're ready to move onto Installing PHP4!

back to top of page

Installing PHP4

PHP is a widely-used general-purpose scripting language that is especially suited for web development and can be embedded into HTML. PHP4 can be downloaded from http://www.php.net/downloads.php/.

Scroll down the Downloads page and download the following files to your hard drive (it's best to create a new program folder for this, e.g. C:\Programs\PHP4):

  • PHP 4.4.1 zip package [8,082Kb] - 31 October 2005 (CGI binary plus server API versions for Apache™, Apache™ 2 (experimental), ISAPI, NSAPI, Servlet and Pi3Web. MySQL support built-in, many extensions included, packaged as zip)

and

  • PHP 4.4.1 installer [1,110Kb] - 31 October 2005 (CGI only, MySQL support built-in, packaged as Windows installer to install and configure PHP, and automatically configure IIS, PWS and Xitami, with manual configuration for other servers. N.B. no external extensions included)

To begin the installation of PHP we need to extract the contents of the ZIP file, php-4.4.1-Win32.zip from C:\Programs\PHP4 (or the drive/path you have downloaded to) to C:\PHP.

A subdirectory, php-4.4.1-Win32 is automatically created. The contents of this directory need to be moved up a level in the file system, so that php.exe is stored in C:\PHP.

Then start the installation file, php-4.4.1-installer.exe:

The following dialog offers the options, Standard and Advanced; select Standard if you are not yet a PHP expert!

Important! In order to avoid problems at a later stage, please accept all default paths and directories proposed by the Installer!

If wished, you may alter the SMTP server address and the From address for the mail function; this is however not compulsory.

In the Server Type dialog select the server type Apache™:

Following completion of the PHP installation, a message box appears, informing that there is no automatic configuration of the Apache™ server, and that it needs to be altered manually. However as we already did this following installation of the Apache™ server (remember adapting the httpd.conf file?), this message can be ignored!

Finally an informational window appears, relevant to Windows NT users, who may encounter problems with permissions for various PHP files and directories:

If this is relevant then please read carefully! Otherwise simply click the OK button to end the installation.

back to top of page

Configuring PHP4

Following installation of PHP, the PHP parser now needs to be configured. The php.ini-recommended file, found in the PHP main directory, needs to be moved to C:\Windows and renamed php.ini. Should you experience any problems with this, please download the php.ini file found at http://www.ibexpert.com/download/ds2/ (ibebenchmarks.zip). If a php.ini file already exists in this directory, please delete before renaming the php.ini-recommended file. You also need to ensure that there is no longer a php.ini in the C:\PHP directory, where the php.exe file is. If you accidentally copied the file instead of moving it, you will now need to delete it.

Important!

The PHP parser is configured by the php.ini. This file is used for the specification of many parameters and [Variable | variables]]. The php.ini also includes a multitude of explanations and examples, recognized by the semicolon comment markers placed at the beginning of each line.

The following variables need to be modified or commented out in the php.ini:

Parameter=ValueAction />Previous valueComment
max_execution_time = 300alter value30 
max_input_time = 120alter value60 
memory_limit = 64Malter value8M 
error_reporting = E_ALL & ~E_NOTICEalter valueE_ALLImportant: Modify the last error_reporting entry in the php.ini! Many are commented out by semicolons.
display_errors = Onalter valueOff 
magic_quotes_sybase = Onalter valueOff 
extension_dir = "c:/php/extensions"alter value"./" 
upload_max_filesize = 32Malter value2M 
extension=php_interbase.dlluncomment (i.e. remove semicolon) This entry is usually commented out. it can be found among the many other extension parameters.

Finally take a quick look to check that the php_interbase.dll really is under C:\php\extensions\ and, of course, restart the Apache™ server before continuing.

We should now test the Apache™ server, to see how successful we have been so far!

Let's create a pseudo php file. Go to C:\htdocs, right-click, select New - Text document, and save this with the .PHP-suffix, for example, TEST.PHP. Now enter the following:

Important: Do not forget to restart the Apache™ server following any changes to any relevant files!

Then simply enter the following in your browser:

 http://localhost 

(or click the link!). If you're successful so far, you should be able to view the following on your screen:

and when you type

 http://localhost/test.php

it should look something like this:

(The illustration shows that the InterBase® module is integrated.)

Trouble-shooting

If you not have been successful, try the following:

  • Open the Apache Services Monitor and Restart the Apache™ server.

Problems with the Windows php.ini file?:

  • Reinstall the Installer
  • Do not write over the default C:\Windows\php.ini file, but simply leave it as it is.

So now we've successfully installed the Apache™ server and PHP, it's time to install the Firebird database and IBExpert.

back to top of page

Installing Firebird 1.5

For details regarding the installation of open source database Firebird, please refer to download and install Firebird.

Installing Firebird .NET Data Provider

As the drivers, ds2webdriver.exe and ds2firebird.exe, have been developed in C#, which is a pure language (.NET is a runtime environment for such programming languages such as C#, J# and VisualBasic.), you will also need to install the Firebird .NET Data Provider following the installation of Firebird itself.

The Firebird .NET Data Provider enables Firebird databases to be accessed from applications which run in a .NET environment.

The Firebird .NET Data Provider can be downloaded from http://www.firebirdsql.org/.

Go to the Download page and click on Firebird .NET Data Provider. Under the heading, Version 1.7a for .NET 1.1 and Mono:: 1.1, select the first file, 7th June 2005 - Data Provider for .NET Framework 1.1 (.exe) (902kb), then a mirror, and then click the icon on the right in the Download column to save the file to your hard drive.

To install after downloading, execute the downloaded file, select the installation language of your choice and work through the usual Install Wizard prompts.

Trouble-shooting

If you receive an error message, when trying to install the FirebirdNETProvider-1.7a-NET1.1.exe, informing you that you cannot install because the Microsoft .NET Framework Version 1.1 ist not installed on your computer, you can download this from http://www.microsoft.com/downloads/.

back to top of page

Installing IBExpert

If you do not yet have a full IBExpert version installed, the IBExpert Personal Edition is also sufficient for creating the database and filling it with data. The IBExpert software can be downloaded from http://www.ibexpert.com/download/.

For details regarding the download and installation of all IBExpert versions, please refer to the relevant chapters in the online documentation?.

back to top of page

Creating the DVD Store database

Depending on which of the IBEBenchmark solutions you have decided upon (full web application simulation or database performance test) you now need to create the DVD Store database (DS2) and fill it with data.

Download DS2.ZIP from http://www.ibexpert.com/download/ (found in the ds2 directory) and save the file to the hard drive's root directory (C:\). Extract all files to the root directory (ensuring the Winzip options, All files/folders in archive and User folder names are checked in the Extract dialog).

A new folder, C:\DS2\ should now be present on your hard drive, with the following structure:

You then need to work through the following:

Preparing the web interface

To prepare the web interface, go to the root directory (C:\htdocs) of your web server and create a subdirectory DS2 (e.g. C:\htdocs\ds2\). This will be the virt_dir for the test run.

Move all files from C:\DS2\firebirdds2\web\php to the folder in your root directory of the web server, e.g. C:\htdocs\ds2.

Creating data for the database

There are three sets of data for customers, orders and products, in three sizes:

DatabaseSizeCustomersOrders (monthly)ProductsCreation period -loading into the DB-
Small10 MB20,0001,00010,000ca. 5 minutes
Medium1 GB2,000,000100,000100,000a few hours
Large100 GB200,000,00010,000,0001,000,000not tested

The procedure for creating the three types of data is the same. Go to C:\DS2\data_files\cust and execute the create_cust.exe to create the data for the Customer table. Select the desired database size (please pay attention to the duration periods when it comes to loading the data in to the database!) and click the Create Cust button.

After the data has been generated, simply click the Close button, and proceed in the same way with the remaining files:

  • In C:\DS2\data_files\orders execute create_orders.exe (for ORDERS, ORDERLINES and CUST_HIST tables)
    • Select the database size
    • Click on the Create Orders button
  • In C:\DS2\data_files\orders execute create_inv.exe
    • Select the database size
    • Click on the Create Inv button
  • In C:\DS2\data_files\prod execute create_prod.exe for creating and loading the product data
    • Select the database size
    • Click on the Create Prod button

After generating all data, you are now ready to create the database and load the data into it.

back to top of page

Creating the database and loading the data

Once the data has been created, go to C:\ds2\firebirdds2\build and execute the firebirdds2_create_all.sql either:

1. using the IBExpert Script Executive - open the file in the text editor and copy the firebirdds2_create_all.sql contents from the Editor into the Script Executive and execute using the respective toolbar icon or [F9]:

2. or using the IBExpert command-line tool, ibescript.exe; simply use the following syntax:

 IBEScript.exe c:\DS2\firebirdds2\build\firebirdds2_create_all.sql

Depending on the selected database size, this may take some time! (Please refer to Creating data for the database for details.)

When the database has been successfully created and filled with data, you are ready to start benchmark testing!

Note: the DVD Store database is automatically generated with the name DS2.FDB and stored at C:\DS2\. It also has the default Firebird/InterBase® user name: SYSDBA and password: masterkey. This user and password is copied in the drivers so please do not alter!

If you intend to evaluate the database in IBExpert, please don't forget to register the database first! (Please refer to the IBExpert documentation (Register Database) for further information.

back to top of page

Testing the installation

There are two ways of testing the installation:

  • Option 1: accessing the database through a web interface with ds2webdriver.exe or
  • Option 2: accessing the database directly with ds2firebird.exe.

back to top of page

Preparing for the run

Option 1: Working through the web interface

In c:\ds2\drivers open the file named ds2webdriver.conf and set the parameters for the run:

Parameter NameDescriptionDefault Value
config_fileconfig file pathnone
targetdatabase/web server hostname or IP addresslocalhost
n_threadsnumber of driver threads1
ramp_ratestartup rate (users/sec)10
run_timerun time (min) - 0 is infinite0
db_size_strdatabase size (S,M or L)S
warmup_timewarmup_time (min)1
think_timethink time (sec)0.25
pct_newcustomerspercent of customers that are new customers20
n_searchesaverage number of searches per order3
search_batch_sizeaverage number of items returned in each search5
n_line_itemsaverage number of items per order5
virt_dirvirtual directory (for web driver)ds2
page_typeweb page type (for web driver)php

Option 2: Working directly with the database

In c:\ds2\drivers open the file named ds2firebird.conf and set the parameters for the run:

Parameter NameDescriptionDefault Value
targetdatabase/web server hostname or IP addresslocalhost
n_threadsnumber of driver threads1
ramp_ratestartup rate (users/sec)10
run_timerun time (min) - 0 is infinite0
db_size_strdatabase size (S,M or L)S
warmup_timewarmup_time (min)1
think_timethink time (sec)0.25
pct_newcustomerspercent of customers that are new customers20
n_searchesaverage number of searches per order3
search_batch_sizeaverage number of items returned in each search5
n_line_itemsaverage number of items per order5
virt_dirvirtual directory (for web driver)ds2
age_typeweb page type (for web driver)php

back to top of page

Starting the test

Option 1: Working through the web interface

Execute ds2webdriver.exe. You'll find it in C:\ds2\drivers. You will also find the configuration file, ds2webdriver.conf in the same directory, where you can change parameters for the test runs.

Example

 C:\DS2\drivers>ds2webdriver.exe --config_file=ds2webdriver.conf

Option 2: Working directly with the database

Execute ds2firebird.exe. You'll find it in C:\ds2\drivers. You will also find the configuration file, ds2firebird.conf in the same directory, where you can change parameters for the test runs.

Example

 C:\DS2\drivers>ds2firebird.exe --config_file=ds2firebird.conf

Please also refer to ds2driver_doc.txt in c:\ds2\drivers.

back to top of page

Appendix I: Readme files

The Appendix includes the more important readme.txt files included in the DS2.ZIP. The majority of the information has however already been covered in the main section of the IBEBenchmark Installation Notes.

ds2_readme.txt

DS2 comes in 3 standard sizes:

DatabaseSizeCustomersOrders (monthly)ProductsCreation period -loading into the DB-
Small10 MB20,0001,00010,000ca. 5 minutes
Medium1 GB2,000,000100,000100,000a few hours
Large100 GB200,000,00010,000,0001,000,000not tested

Requirements

The full Installation Guide can be read online at: http://ibexpert.net/ibe/pmwiki.php?n=Doc.IBEBenchmarks.

Install notes (German language only) for PHP, Apache™, IBExpert and Firebird can be found at: http://www.phptree.de/documentation/

 /**************************************************/

 1. Copy/Move the complete folder structure to "C:\"  so you 
 have this file at "C:\DS2\DS2-README.txt"


 /******** Preparing the web interface *************/

 2. Create a folder named "ds2" in the root directory (htdocs) of your 
 web server. This will be the "virt_dir" for the test run.

 Copy/Move all files from "\DS2\firebirdds2\web\php\" to the folder 
 "ds2" in your web server's root directory,
         for example to :  c:\htdocs\ds2
 /******* Creating the Data for the Database *******/

 3. in   "c:\ds2\data_files\cust"
  -execute "create_cust.exe"
   - Choose the size of the database
   - click the "Create Cust" button

 4. then in   "c:\ds2\data_files\orders" 
  a - execute "create_orders.exe"
   - Choose the size of the database
   - click the "Create Orders" button
  b - execute "create_inv.exe"
   - Choose the size of the database
   - click the "Create Inv" button

 5. then in   "c:\ds2\data_files\prod"
  - execute "create_prod.exe"
   - Choose the size of the Ddtabase
   - click the "Create Prod" button

 /* Creating the Database and Loading the Data          */
 /*                                                     */
 /*        ATTENTION!!!!!!!!!!!!!!!                     */
 /*                                                     */
 /* there are 3 sizes of the ds2 database.             */
 /*        SIZE            CREATING DURATION            */
 /* -small     10MB            ~ 5 min                  */
 /* -medium     1GB            ~ some hours (~ 5)       */
 /* -large    100GB            ~ NOT TESTED             */
 /*                        could last a very long time  */


 6. In   "c:\ds2\firebirdds2\build" you can find a SQL Script "firebirdds2_create_all.sql"
  execute it with the "IBExpert Script Executive" or its command-line tool "IBEScript.exe"
  (usage:     IBEScript.exe c:\DS2\firebirdds2\build\firebirdds2_create_all.sql)

  (depending on the chosen DB size it can last between some minutes and several hours)

 /*********************************************************/
 *
 *There are two ways of testing:
 *
 * 1. access the database through a web interface
 *    with ds2webdriver.exe
 *  or
 * 2. access the database directly with 
 *    ds2firebird.exe
 *
 /*********************************************************/


 /*********** Preparing for the run ***********************/

 7 a)
  In "c:\ds2\drivers" open the file named "ds2webdriver.conf"
     depending on the type of database, you need to set the parameters for the run:

  Parameter Name      Description                                         Default Value

  target              database/web server hostname or IP address          localhost
  n_threads           number of driver threads                            1
  ramp_rate           startup rate (users/sec)                            10
  run_time            run time (min) - 0 is infinite                      0
  db_size_str         database size (S,M or L)                            S
  warmup_time         warmup_time (min)                                   1
  think_time          think time (sec)                                    0.25
  pct_newcustomers    percent of customers that are new customers         20
  n_searches          average number of searches per order                3
  search_batch_size   average number of items returned in each search     5
  n_line_items        average number of items per order                   5
  virt_dir            virtual directory (for web driver)                  ds2
  page_type           web page type (for web driver)                      php

 7 b)
  In "c:\ds2\drivers" open the file named "ds2firebird.conf"
     depending on the type of database, you need to set the parameters for the run:

  Parameter Name      Description                                         Default Value

  target              database/web server hostname or IP address          localhost
  n_threads           number of driver threads                            1
  ramp_rate           startup rate (users/sec)                            10
  run_time            run time (min) - 0 is infinite                      0
  db_size_str         database size (S,M or L)                            S
  warmup_time         warmup_time (min)                                   1
  think_time          think time (sec)                                    0.25
  pct_newcustomers    percent of customers that are new customers         20
  n_searches          average number of searches per order                3
  search_batch_size   average number of items returned in each search     5
  n_line_items        average number of items per order                   5
  virt_dir            virtual directory (for web driver)                  ds2
  page_type           web page type (for web driver)                      php


 /****************** Starting the Test *****************************/


 8 a) 
  Execute "ds2webdriver.exe". You can find it in "c:\ds2\drivers". 
  Also in "c:\ds2\drivers" you can find a config file: "ds2webdriver.conf". 
  There you can change parameters for the test runs.

  Example:

   C:\DS2\drivers>ds2webdriver.exe --config_file=ds2webdriver.conf

 8 b)
  Execute "ds2firebird.exe". You can find it in "c:\ds2\drivers". 
  Also in "c:\ds2\drivers" you can find a config file: "ds2firebird.conf". 
  There you can change parameters for the test runs.

 Example:

   C:\DS2\drivers>ds2firebird.exe --config_file=ds2firebird.conf

back to top of page

ds2_data_files_readme.txt

Instructions for building DVD Store Version 2 (DS2) database data files.

DS2 comes in 3 standard sizes:

DatabaseSizeCustomersOrders (monthly)Products
Small10 MB20,0001,00010,000
Medium1 GB2,000,000100,000100,000
Large100 GB200,000,00010,000,0001,000,000

The directories below contain programs to build the data files used by all three sizes.

Directory structure

 ./ds2/data_files
 ./ds2/data_files/cust    Customer table
 ./ds2/data_files/orders  Orders, Orderlines and Cust_Hist table
 ./ds2/data_files/prod    Products, Inv table

Please also refer to the READMEs in the other directories.

<dave_jaffe@dell.com> and <todd_muirhead@dell.com> 6/28/05
<asowarenko@ibexpert.biz> 28.10.2005

back to top of page

ds2_drivers_readme.txt

In this directory are the components necessary to create a database-independent web driver as well as instructions for compiling database-dependent direct drivers.

When compiled with the appropriate set of web- or database-dependent functions, ds2xdriver generates orders against the DVD Store Database V.2 through web interface or directly against database. Simulates users logging in to store or creating new customer data; browsing for DVDs by title, actor or category, and purchasing selected DVDs

Compile with appropriate functions file to generate driver for web or Firebird target:

 csc /out:ds2webdriver.exe       ds2xdriver.cs ds2webfns.cs       /d:USE_WIN32_TIMER /d:GEN_PERF_CTRS
 csc /out:ds2firebird.exe        ds2xdriver.cs ds2firebirdfns.cs  /d:USE_WIN32_TIMER /d:GEN_PERF_CTRS

USE_WIN32_TIMER: if defined, program will use high resolution WIN32 timers (not supported in mono)
GEN_PERF_CTRS: if defined, program will generate Windows Perfmon performance counters (not supported in mono)

The database functions files are found in the database directories, i.e. ./ds2/firebirdds2

To see syntax, type program name by itself on a command line.

Directory structure

./ds2/drivers/ds2_drivers_readme.txtthis readme
./ds2/drivers/ds2driver_doc.txtusage of the drivers
./ds2/drivers/ds2firebird.confconfigurations for ds2firebird.exe
./ds2/drivers/ds2firebird.exefirebird driver compiled as above
./ds2/drivers/ds2webdriver.confconfigurations for the web driver
./ds2/drivers/ds2webdriver.exeweb driver compiled as above
./ds2/drivers/ds2webdriver_mono.exeweb driver compiled without defines, for mono
./ds2/drivers/ds2xdriver.csmain driver program
./ds2/drivers/ds2webfns.csweb driver functions

<dave_jaffe@dell.com> and <todd_muirhead@dell.com> 6/28/05
<asowarenko@ibexpert.biz> 28.10.2005

back to top of page

ds2driver_doc.txt

Syntax

 ds2webdriver --parameter=value ...       
 ds2firebird --parameter=value ...

Enter parameters with format --parm_name=parm_value
And/or use a config file with argument --config_file=(config file path)
Params will be evaluated left to right.

Parameter NameDescriptionDefault Value
config_fileconfig file pathnone
targetdatabase/web server hostname or IP addresslocalhost
n_threadsnumber of driver threads1
ramp_ratestartup rate (users/sec)10
run_timerun time (min) - 0 is infinite0
db_size_strdatabase size (S,M or L)S
warmup_timewarmup_time (min)1
think_timethink time (sec)0.25
pct_newcustomerspercent of customers that are new customers20
n_searchesaverage number of searches per order3
search_batch_sizeaverage number of items returned in each search5
n_line_itemsaverage number of items per order5
virt_dirvirtual directory (for web driver)ds2
page_typeweb page type (for web driver)php

Typical run

 C:\ds2\drivers> ds2webdriver.exe --target=rhel32 --run_time=1
 target= rhel32 n_threads= 1 ramp_rate= 10 run_time= 1 db_size_str= S warmup_time= 1 think_time= 0.25
 pct_newcustomers= 20 n_searches= 3 search_batch_size= 5 n_line_items= 5 virt_dir= ds2 page_type= php

Using WIN32 QueryPerformanceCounters for measuring response time

 Performance Counter Category Test and Counter MaxRT exist
 Thread 0: created for User 0
 Thread 0: connected to rhel32
 Controller (6/30/2005 9:25:52 PM): all threads running
 Controller (6/30/2005 9:25:52 PM): all threads connected - issuing Start
 et= 10.1 n_overall=23 opm=136 rt_tot_lastn_max_msec=867 rt_tot_avg_msec=182 rollbacks: n=0 %= 0.0
 et= 20.3 n_overall=47 opm=138 rt_tot_lastn_max_msec=867 rt_tot_avg_msec=172 rollbacks: n=0 %= 0.0
 et= 30.5 n_overall=72 opm=141 rt_tot_lastn_max_msec=867 rt_tot_avg_msec=165 rollbacks: n=0 %= 0.0
 et= 40.6 n_overall=99 opm=146 rt_tot_lastn_max_msec=867 rt_tot_avg_msec=160 rollbacks: n=1 %= 1.0
 et= 50.8 n_overall=123 opm=145 rt_tot_lastn_max_msec=723 rt_tot_avg_msec=153 rollbacks: n=1 %= 0.8
 et= 61.0 n_overall=148 opm=145 rt_tot_lastn_max_msec=723 rt_tot_avg_msec=149 rollbacks: n=1 %= 0.7
 Stats reset
 et= 10.1 n_overall=20 opm=118 rt_tot_lastn_max_msec=808 rt_tot_avg_msec=226 rollbacks: n=2 %= 10.0
 et= 20.3 n_overall=44 opm=130 rt_tot_lastn_max_msec=946 rt_tot_avg_msec=186 rollbacks: n=2 %= 4.5
 et= 30.5 n_overall=71 opm=139 rt_tot_lastn_max_msec=946 rt_tot_avg_msec=153 rollbacks: n=2 %= 2.8
 et= 40.6 n_overall=102 opm=150 rt_tot_lastn_max_msec=946 rt_tot_avg_msec=133 rollbacks: n=3 %= 2.9
 et= 50.8 n_overall=123 opm=145 rt_tot_lastn_max_msec=1205 rt_tot_avg_msec=139 rollbacks: n=4 %= 3.3
 et= 60.9 n_overall=150 opm=147 rt_tot_lastn_max_msec=1205 rt_tot_avg_msec=137 rollbacks: n=5 %= 3.3
 Run over
 Final: et= 61.0 n_overall=150 opm=147 rt_tot_lastn_max=1205 rt_tot_avg=137 n_login_overall=121  

 n_newcust_overall=29 
 n_browse_overall=403 n_purchase_overall=150 rt_login_avg_msec=33 rt_newcust_avg_msec=21 
 nt_browse_avg_msec=9 
 rt_purchase_avg_msec=80 n_rollbacks_overall=5 rollback_rate = 3.3%
 Thread 0: exiting
 Controller: all threads stopped, exiting; n_purchase_from_start= 299 n_rollbacks_from_start= 6
 C:\ds2\drivers>

Description of counters

et: elapsed time (since start if during warmup; since Stats reset if after warmup time is over)
n_overall: total number of orders processed between Stats reset and Run over
opm: orders per minute (exported to Perfmon counter test/opm)
rt_tot: total response time seen by a user during an order cycle (sum of all response times during order: login/create new customer, browse(s), purchase)
rt_tot_avg_msec: average of rt_tot over all orders in milliseconds
rt_tot_lastn_max_msec: maximum rt_tot seen by last 100 users in milliseconds (exported to perfmon counter test/rt)
rollbacks: number of time order has to be rolled back due to insufficient quantity (these are counted in the opm but won't show up as a new Order in the database)

Final (all stats are accumulated between Stats reset and Run over):
n_login_overall: total number of logins during run
_newcust_overall: total number of new customer registrations during run
_browse_overall: total number of browses during run
_purchase_overall: total number of purchases during run
rt_login_avg_msec: average login time in milliseconds
rt_newcust_avg_msec: average new customer registration time in milliseconds
rt_browse_avg_msec: average browse time in milliseconds
rt_purchase_avg_msec: average purchase time in milliseconds
_rollback_overall: total number of rollbacks during run
rollback rate: n_rollback_overall/n_overall * 100%

After all threads are stopped, the total n_purchase and n_rollbacks during the entire run, including warmup and while the threads are stopping, is printed out the number of new orders in the database during the run should be n_purchase_from_start - n_rollbacks_from_start.

back to top of page

ds2_firebird_build_readme.txt

Instructions for building DVD Store Version 2 (DS2) database.

Execute with the IBExpert Script Executive or with IBEScript:

 firebirdds2_create_all.sql

ds2_firebird_web_readme.txt

The Firebird DVD Store currently has one web application interface: PHP

For the ds2webdriver program and source, see ./ds2/drivers.

'Directories

./ds2/firebirdds2/web
./ds2/firebirdds2/web/php php pages

<asowarenko@ibexpert.biz> 28.10.2005

back to top of page

ds2_firebird_web_php_readme.txt

PHP interface to the Firebird DVD Store database Requires php4 / Firebird 1.5 / Apache™

Files

dscommon.inc
dspurchase.php
dsnewcustomer.php - new customer page with stored procedure call
dsnewcustomer.php.sp - new customer page with stored procedure call
dsnewcustomer.php.nosp - new customer page without stored procedure call
dsbrowse.php
dslogin.php

The driver programs expect these files in a virtual directory ds2. In your web server you either need to create a virtual directory that points to this directory or copy these files to the appropriate directory (eg c:\htdocs\ds2)

<asowarenko@ibexpert.biz> 28.10.2005

back to top of page
<< | IBExpert DemoDB | >>