Storing mnoGoSearch indexer data in ODBC data sources

This article describes how to use mnoGoSearch with an Easysoft ODBC driver.

Easysoft ODBC drivers have been tested with mnoGoSearch 3.0.10pre, 3.0.10 (which includes patches submitted by Easysoft for ODBC) and all versions up to 3.2. 3.0.10pre was a pre-release of UDMSearch in which the mnoGoSearch development team added the --with-easysoft configuration option. When Easysoft tested 3.0.10pre of mnoGoSearch with ODBC-ODBC Bridge and SQL Server a few problems were found, which should all be fixed by the time you read this (refer to Notes on 3.0.10pre below). As a result, Easysoft only recommend running 3.0.10 or later and not the prerelease version.

Contents

Configuring and building

Read the INSTALL file in the mnoGoSearch distribution first. To use multiple ODBC drivers with mnoGoSearch, build mnoGoSearch with unixODBC (--with-unixODBC) and then install the Easysoft ODBC driver under unixODBC.

Older versions of mnoGoSearch that predate unixODBC do not have the --with-unixODBC configuration option. You need to build mnoGoSearch with the --with-easysoft option instead. Note that if you build mnoGoSearch with --with-easysoft on the configure line, it can then only be used with the ODBC-ODBC Bridge.

Follow the instructions in the mnoGoSearch INSTALL file using either --with-unixODBC or --with-easysoft on the configure line. Once built correctly and installed, you need to follow the instructions in mnoGoSearch's doc directory to configure the indexer and search.cgi scripts.

Once this is complete, you must ensure that you create the mnoGoSearch tables in your database. mnoGoSearch contains subdirectories under create in the distribution tree for each database engine. For example, create/mssql/create.sql contains the text to create the mnoGoSearch tables for SQL Server (refer to the mnoGoSearch documentation). You have these alternatives:

  1. Copy this file to your database server and run it through your database engine.
  2. Define a data source for your Easysoft ODBC driver (an example is shown below) and then create the tables in your database by using the Easysoft ODBC driver. You can use command line utilities like unixODBC's isql to do this, or write a simple piece of C code that does an SQLExecDirect on each of the SQL statements (in this case, delete the comments and GO commands).

Now create a data source that unixODBC can load.

If you built mnoGoSearch with unixODBC, run unixODBC's ODBCConfig program and create an Easysoft ODBC driver data source. If you did not build unixODBC with the GUI options, you will have to edit your ~/.odbc.ini or system odbc.ini file manually to add the data source.

If you built mnoGoSearch with the ODBC-ODBC Bridge directly, you must create a data source in any of the places the ODBC-ODBC Bridge searches for DSNs (these include the ODBCINI environment variable, ./odbc.ini, ./.odbc.ini, ~/odbc.ini, ~/.odbc.ini, and /etc/odbc/ini. Consult the ODBC-ODBC Bridge documentation or knowledge base for more information.

The following example shows a sample ODBC-ODBC Bridge client DSN:

[dsn_name]
Description = mnoGoSearch
Driver = ODBC-ODBC Bridge
ServerPort = machine_where_ODBC-ODBC Bridge_server_installed:8888
TargetDSN = system_dsn_on_server_machine
LogonUser = server_OS_user_name
LogonAuth = server_OS_password
TargetUser = user_name # (optional)
TargetAuth = password # (optional)

For information about what each attribute means, refer to the ODBC-ODBC Bridge documentation. For information about adding a data source for a different Easysoft ODBC driver, refer to the documentation for your Easysoft ODBC driver.

Once the data source is defined and you have edited your indexer.conf file, you should be able to run the mnoGoSearch indexer command, which writes records about the web and FTP sites you have defined to the database.

Don't forget to make sure the data source is also available to your web server. Otherwise, when you run the mnoGoSearch search.cgi program to search the indexed pages it will fail to connect to your ODBC database.

Notes on 3.0.10pre

If you're thinking about using mnoGoSearch 3.0.10pre with the ODBC-ODBC Bridge and SQL Server, Easysoft recommend looking for a more recent version. 3.0.10pre (which is the first version we tested) has a few problems that affect the ODBC-ODBC Bridge and SQL Server:

  1. The create.sql file in create/mssql defines the lang column of the url table as being NOT NULL. This should be changed to NULL.
  2. There is no easy way in UNIX to get the SQL in create.sql into SQL Server. You can copy create.sql to the machine where SQL Server is located and do it there. Otherwise, you need to write a small ODBC program (or use the unixODBC isql program) that calls SQLExecDirect for each statement in create.sql.
  3. The quoting rules used by SQL Server are used by mnoGoSearch. Specifically, the single quote character (') is escaped by prefixing with backslash (\) instead of using two single quotes (''). This may be temporarily fixed by editing sql.c and making the following change:

    In the escstr function for non-MYSQL and Oracle databases, change the action of the '\'' and '\\' cases from:

    *x='\\';x++;

    to:

    *x='\'';x++;

    The proper fix for this would be to add a new DBType indexer configuration for SQL Server and this has been done in later versions.

    Without this change, you may get syntax errors when running mnoGoSearch's indexer through the ODBC-ODBC Bridge to SQL Server.