Building the Server

From DSP Wiki
Jump to: navigation, search

Installing & Setting up the DS Server on Windows

Prerequisites

In this section we will download every program/tool we will need to setup and maintain the servers. Do not install these yet.

  • Ability to interact with said database, one of:
    • MySQL Workbench
    • HiediSQL (comes with MariaDB if you used that, and there are also portable editions)
    • Any other database management GUI or learn to command line (its super easy and you can google your way through it)

Installation

IMPORTANT NOTICE!

There may be certain issues when trying to use newer versions of MySQL or MariaDB. Pull requests to remedy these issues are welcome.

Some feature change:

  • The PASSWORD() function is removed in the latest version of MySQL
  • Use of the VALUES() function is limited to INSERT ... ON DUPLICATE KEY UPDATE in the latest versions of MariaB and MySQL
  • The latest MySQL may complain about default timestamps on certain tables - this is also a known issue with using XAMP distributions in some configurations.

Recommended versions are:

  • MariaDB 10.4
  • MySQL Community Server 5.7
  • MySQL Community Server 5.6

MariaDB

It's pretty straightforward, the defaults are fine and you can optionally change the root password (but better yet don't use the root user for everything). The same is true for MySQL.

MySql

(flag)

Choose Typical Installation

After the main installation, select Launch the MySQL Instance Configuration Wizard

Select Detailed Configuration

Select Developer Machine

Select Multi-functional Database

Standard Installation Directory

Select Manual Setting and 1000 Concurrent connections

Select Enable TCP/IP Networking, Port Number 3306, and Enable Strict Mode

Select Best Support for Multilingualism

Select Install as Windows Service, Launch the MySQL Server automatically, and Include Bin Directory in Windows PATH.

Select Modify Security Settings, and enter a new password twice. Write down the MySQL password. (DO NOT LEAVE BLANK.)

Select Next, Execute, then Finish

MySQL 5.6 (Updated)

Note: Server is x86, but will install 32-bit and 64-bit binaries.

Accept License Terms

Skip Check for updates

Select Developer Default

Check requirements and click Execute - will launch installers for anything you're missing

Default Installation - ~5min

Default Configuration - Check box for "Show Advance Options" - Click Next

Enter MySQL Root Password, twice. Remember it! - Click Next

It will show windows service details, leave as default and Click Next

It will show Logging options, leave as default and Click Next

Configuration will complete, Click Next

Samples and examples will configure - ~5min - Click Next

Click Finish

MySQL 5.7 (New)

Note: Server is x86, but will install 32-bit and 64-bit binaries.

Accept License Terms

Choosing setup type: Keep defaults. Note, you can select "Server only", but Developer Default will allow you to skip the MySQL Workbench below, and it will require more diskspace.

Check Requirements: Click next. You should not need anything that shows "Manual", but you can download the modules separately if you do need them later for some reason. *Note, if you are missing some of the elements (Status = manual), it will pop up with a message stating "One or more product requirements have not been satisfied". Click yes to continue.

Installation: It will show a list of those things that you will install (it will not list those that had "Manual" in previous step). Click Execute.

Product Configuration: This screen will just tell you that there is still some stuff to configure. Click Next.

Group Replication: Leave default setting (Standalone MySQL Server / Classic MySQL Replication) and click Next

Type and Networking: Leave default settings and click Next (Only reason you would need to change is if you are planning to run a dedicated online server, in which case, I hope you know what these options already mean...)

Accounts and Roles: Enter MySQL Root Password, twice. Remember it! - Click Next

Windows Service: Leave default settings and click Next.

Plugins and Extensions: Leave default settings and click Next.

Apply Configuration: Click Execute. It will show you the progress as it configures your MySQL installation. Once complete, click Finish

Product Configuration: Click Next, (This will set up some samples and examples databases)

Connect to Server: Enter the password you entered for the MySQL (Under Accounts and Roles section) at the bottom where it asks for password, then click Check. As long as the password is correct, it will ungrey the Next button and allow you to continue. Click Next.

Apply Configuration: Just like the Apply configuration above. Click Execute, it will configure, then click Finish once complete.

Product Configuration: Click Next.

Installation Complete: As per the title, you are done. Click Finish and enjoy your new MySQL database.


MySQL Workbench

Use default installation options. This can still be installed and will still function even if you use MariaDB instead of stock MySQL. It will generate a few warnings from by a version number check that you can safely ignore.

Visual Studio 2019

Any edition works. Community is free.

Make sure you install Desktop development with C++ when selecting development packages.

Git for Windows

Use default installation options.

TortoiseGit

Use default installation options. You can skip the step to reboot since we'll reboot once everything else is installed.

Visual C++ Redistributable Runtime 2015

Use default installation options.

.Net 4.6 (or higher)

Use default installation options. It probably comes with your Visual Studio install anyway. Dot Net Frameworks always come with all prior versions packed in, so you never need to install multiple versions.


At this point everything should be installed that needs to be and you can restart your PC to finalize them.

Downloading the Server Source Code

In this section we will use TortoiseGit to download the latest git source code for the server.

Open Windows Explorer (right click the start button and choose Windows Explorer)

Right click on any folder select Git Clone

In the "URL" field put in the following url:

http://github.com/DarkstarProject/darkstar.git

Set the "Directory" to your choice (blank will default to a new folder named "darkstar" at the current location)

Click OK.

Wait for the download to finish.

Optional: pull submodule to get navmeshes [1]

Click OK to close TortoiseGit.

Close windows explorer

At this point you have downloaded all of the source code for the latest version.

If you are not planning on doing any development, it's advised to use the stable branch instead of master. Right click on your darkstar folder, and go to TortoiseGit -> Switch/Checkout. Change the branch dropdown to remotes/origin/stable and hit OK.

Setting up the Database

In this section we will use the data that you downloaded to build a database in MySQL.

Using a bat script

If you follow this step, you may skip the next one.

Paste the following into notepad and save it as DSP_Import.bat in your dsp\sql folder:

@ECHO OFF
REM =============================================================================
REM =============================================================================
REM ======                                                              =========
REM ====== This script will drop the DB specified, then create the DB  =========
REM ====== specified, and then load all .sql tables from its run dir to =========
REM ====== the the DB.                                                  =========
REM ======                                                              =========
REM ====== File needs to be run from within the \dsp\sql folder (same   =========
REM ====== folder with all the .sql files. Please edit as needed. By    =========
REM ====== default it WILL DROP the standard dspdb DB, losing all       =========
REM ====== accounts and characters. If this is not desired, then update =========
REM ====== the file to load the new DB into a new DB name.              =========
REM ======                                                              =========
REM ====== Update -p with MySQL password. If you password is 'foo',     =========
REM ====== then change '-pMYSQLPASS' to '-pfoo' (3 places).             =========
REM ======                                                              =========
REM ====== If you want to use a different database name, change 'dspdb' =========
REM ====== with a database name of your choosing.                       =========
REM ======                                                              =========
REM =============================================================================
REM =============================================================================

ECHO Creating Database dspdb
"c:\edit\this\path\bin\mysqladmin" -h localhost -u root -pMYSQLPASS DROP dspdb

ECHO Creating Database dspdb
"c:\edit\this\path\bin\mysqladmin" -h localhost -u root -pMYSQLPASS CREATE dspdb

ECHO Loading dspdb tables into the database
cd c:\path\to\darkstar\sql
FOR %%X IN (*.sql) DO ECHO Importing %%X & "c:\edit\this\path\bin\mysql" dspdb -h localhost -u root -pMYSQLPASS < %%X

ECHO Finished!

At a minimum you will need to change -pMYSQLPASS in the script to where root is your mysql root password. If your password is 'foo' then you would use '-pfoo' instead. You may also have to change the path of your mysql installation as well (for example, if you installed a different version of mysql)

You can use this script any time the repository is updated, but it will drop the table and thus delete all your player data. You may want to make manual changes instead unless you are using it purely as a test server.

After you've done your initial load, you can use the following script that will preserve character data

cd c:\darkstar\sql
del auction_house.sql
del chars.sql
del accounts.sql
del accounts_banned.sql
del char_effects.sql
del char_equip.sql
del char_exp.sql
del char_inventory.sql
del char_jobs.sql
del char_look.sql
del char_pet.sql
del char_pet_name.sql
del char_points.sql
del char_profile.sql
del char_skills.sql
del char_spells.sql
del char_stats.sql
del char_storage.sql
del char_vars.sql
del char_weapon_skill_points.sql
del chars.sql
del conquest_system.sql
del delivery_box.sql
del linkshells.sql
FOR %%X IN (*.sql) DO ECHO Importing %%X & "c:\edit\this\path\bin\mysql" dspdb -h localhost -u root -pMYSQLPASS < %%X

Using HeidiSQL

Configure Heidi to use your MariaDB or MySQL connection. Read the documentation or click "new" and wing it like those brave souls who pretend they understand ikea manuals.

Open the "session" then right click it and move your cursor to "new" so the submenu opens and click "database". All instructions after this point will assume you named your database "dspdb" without quotes.

Now Heidi needs to know which Database we are loading tables into so click to select your new database so it gets higlighted.

Now select "file" again at the top left of the window.

Select "Load SQL file" (or press control+O and that's the letter not zero).

Browse to your SQL directory in your repository root and pick a file you want to import. Like C:\DSP\sql\abilities.sql for example. You can also hold control to select multiple files at once, but be aware Heidi will open a new tab for every file. Press F9 (or click the blue right facing arrow icon) to execute or "import" the current tab's SQL content into your database. Repeat until all tables and triggers have been loaded.

That's it, your are done loading tables.

Using MySQL Workbench

Todo!

Using Navicat

NOTE: The free edition of Navicat was discontinued ages ago. No more support or bugfixes are released for it. And even if you use the latest commercial release, it will do annoying things like surround every field of a dump in single quotes. If you pull request an sql change that places single quotes on number fields on 30,000 lines Teo reserves the right to maim you. Just kidding. Or am I? :) Alternatives include Heidisql, mysql workbench, and many others that are free and open sourced. Same functionality, different layout on screen.

Click Connection (or File, New Connection)

Connection name: DarkStar

Host Name/IP: localhost

Port: 3306

User Name: root

Password: Enter the password you wrote down during the mySQL installation.

Check Save Password

Click OK


In the connections pane double click DarkStar

Right click on DarkStar and select New Database...

Name it dspdb

Use the Default character set

Select OK.


Double click dspdb

Right click on dspdb and select Execute SQL File...

Browse to C:\dsp\sql and select abilities.sql

Click Start

Continue this process for each file in the folder.


If you have an issue with the triggers.sql file then follow this step. If not, proceed to the next step (IP address for the map server):


In Navicat Open accounts table

Click File, Design Table

Click the Triggers tab

There should a line already listed with account_delete in the Name field


If not:

click Add Trigger

Name column should be account_delete

Fires column should be Before

Check the box in the Delete column

In the definition box below add the following code:

   DELETE FROM `accounts_banned` WHERE `accid` = OLD.id;
   DELETE FROM `chars` WHERE `accid` = OLD.id;

Select Save and then close the table.

Open the chars table

Click File, Design Table

Click the Triggers tab

There should already be a line listed with char_delete in the name field.


If not:

click Add Trigger

Name column should be char_delete

Fires column should be Before

Check the box in the Delete column

In the definition box below add the following code:

   DELETE FROM `char_exp` WHERE `charid` = OLD.charid;
   DELETE FROM `char_jobs` WHERE `charid` = OLD.charid;
   DELETE FROM `char_equip` WHERE `charid` = OLD.charid;
   DELETE FROM `char_effects` WHERE `charid` = OLD.charid;
   DELETE FROM `char_look` WHERE `charid` = OLD.charid;
   DELETE FROM `char_stats` WHERE `charid` = OLD.charid;
   DELETE FROM `char_skills` WHERE `charid` = OLD.charid;
   DELETE FROM `char_titles` WHERE `charid` = OLD.charid;
   DELETE FROM `char_inventory` WHERE `charid` = OLD.charid;
   DELETE FROM `char_vars` WHERE `charid` = OLD.charid;


Select Save and then close the table.


Configure the ZoneIP for the Map Server

Note: These instructions largely assume you are using HeidiSQL

Now you need to determine what IP address that will be used for the Map Server, then update MySQL with it.

Most devs & enthusiasts will not have a separate map server, but will run both map server and login server on the same PC.


1. Determine what IP address will be used for the map server.

If the server will be public, then use your external IP in the next step. (Looks really random like 173.194.43.104)

If the server will be internal and you want it available on other PCs in your internal network, use the internal network IP (192.168.x.x) in the next step.

If you only plan to run the client and the server on the same PC, you can use the localhost address (127.0.0.1) in the next step.

2. Update the zone_settings table with the ip (flag) In HeidiSQL, click on the database dspdb and expand

Click on Query tab

Add the following code into the empty space, inserting the value calculated above. The value in the code below (127.0.0.1) would be used if only running the client from the same PC as the server.

    UPDATE zone_settings SET zoneip = '127.0.0.1';

Click on the "run" button (blue arrow), or press F9. You should get a message stating that many (280+) records were changed.

I recommend you save the Query in case you need to run it again later after updating the server.

Close HeidiSQL.

Building the Servers

In this section we will be building the Dark Star servers.

Dark Star uses 3 separate servers. A Login server, a Map/Game server, and an optional Search/Auction House server. Using the ZoneIP mentioned earlier, every map could run on its own server at a different IP address. If you have 4 computers each computer could run different areas, as long as they were all connected to the same MySQL database server.

Build the Server

Open Visual Studio

Click File, Open, Project/Solution (or Ctrl+Shift+O)

Navigate to C:\dsp\win32

Select darkstar.sln

It may take a few minutes to load files.

Optional: if your cpu can handle it, enable multi-core compiling like so http://i.imgur.com/GCjdsdG.png (do this for all 3, or just DSGameServer since that's where most the compile time goes).

Click Debug then Build Server (or Ctrl+Shift+B)

Building will take a few minutes.

Once complete, validate that there are no Errors listed in the summary line at the bottom. It should look like this:

========== Build: 3 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

If there are any SQL errors it is probably caused by the SQL dev package was extracted to the wrong folder.

If there is a #include error, change that #include to say #include <mysql/mysql.h>

When all of the Servers are built

Verify in the C:\dsp directory that the servers you built are there:

DSConnect-server.exe for the Login Server

DSGame-server.exe for the Map Server

DSSearch-server.exe for the Search Server



Configuring the Servers

Now you will configure the Login Server and Map Server to connect to the MySQL database.

Do not mess with any other settings in these files unless you know what they are for and what they do.


Login Server Settings

In windows explorer navigate to c:\dsp\conf

Open login_darkstar.conf in a text editer (notepad)

Scroll down to the bottom and validate/update the following lines:

mysql_host: 127.0.0.1
mysql_port: 3306
mysql_login: root
mysql_password: root <-- change this to your password you wrote down during the MySQL install (flag)
mysql_database: dspdb

Map Server Settings

In windows explorer navigate to c:\dsp\conf

Open map_darkstar.conf in a text editer (notepad)

Scroll down to the bottom and validate/update the following lines:

mysql_host: 127.0.0.1
mysql_port: 3306
mysql_login: root
mysql_password: root <-- change this to your password you wrote down during the MySQL install (flag)
mysql_database: dspdb

Search Server Settings

In windows explorer navigate to c:\dsp\conf

Open search_server.conf in a text editer (notepad)

Scroll down to the bottom and validate/update the following lines:

mysql_host: 127.0.0.1
mysql_port: 3306
mysql_login: root
mysql_password: root <-- change this to your password you wrote down during the MySQL install (flag)
mysql_database: dspdb


At this point we have configured the servers to connect to the SQL database.



Starting the Servers

In this section we will launch our new servers.

Navigate to c:\dsp

Run DSConnect-server.exe as Admin

Run DSGame-server.exe as Admin

(Optional) Run DSSearch-server.exe as Admin

At this point if your servers fail to load, you probably missed a step. Go back through the steps and try to find out what you missed. It may also be helpful to look at the log files located in c:\dsp\log.


Forward Ports to Allow Others to Join (& Firewall)

In this section we will provide the port numbers that need to be forwarded to the server. This is only needed to allow people outside your local intranet to join your server.

In your router and firewall the following ports need to be opened/forwarded to your server:

TCP Ports:
54230
54231
54001
54002

UDP Port:
54230

If still having trouble, disable any firewall software completely for testing. This includes the Windows firewall that comes on all modern Windows systems, which can be found in the Control Panel.

Installing & Setting up the DS Server on Linux

The server is known to build in recent Ubuntu versions (14.04), and is tested regularly against it.

Generic build instructions for most POSIX (GNU/Linux, and possibly FreeBSD, NetBSD, UNIX or even OSX with some additional work) systems follow. Some knowledge of how to compile software on your distribution is assumed. You may find it helpful to reference the Windows guide above if something is unclear.

Prerequisites:

Install on Ubuntu

sudo apt-get install git mysql-server libmysqlclient-dev libluajit-5.1-dev libzmq3-dev autoconf pkg-config

Ubuntu 14.04 needs gcc 7.

sudo su -
apt-get update 
apt-get install -y software-properties-common
add-apt-repository -y ppa:ubuntu-toolchain-r/test
apt-get update
apt-get install -y luajit-5.1-dev libzmq3-dev g++-7 mysql-client-core-5.6 mysql-server-5.6 mysql-client-5.6
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 90

Other Distros

A compiler with C++17 support. Currently tested with GCC and G++, and requires a minimum version of 7

Git to retrieve the source code.

Autotools (automake autoconf m4sh) and pkg-config to generate the configure script and Makefile

mysql server

mysql client library (select the developer version if your distro has on)

luajit library version 5.1 or higher (select the developer version if your distro has one)

libzmq (select the developer version if your distro has one)

If you plan to run from a console, you may also wish to install "screen" as there is no daemon mode for the servers yet.

Preparing the Source Code

Once you have everything mentioned above installed, grab the latest source from the repository via the command:

git clone http://github.com/DarkstarProject/darkstar.git/

If you are not planning on doing development, it's advised to checkout the stable branch:

git checkout stable

Compiling

Next is a typical autotools build. Run the autogen script, followed by configure, and then make (GNU make: if your cpu has multiple cores and you wish to utilise them to speed up the build, use make -j#CoresMultipliedBy1.5 e.g. make -j4 if it has 3 cores instead of just make in the step below):

sh autogen.sh
./configure --enable-debug=gdb
make

You can choose not to configure with debug enabled, but there is very little potential gain in this and it prevents many useful break statements from triggering. The Makefile also has separate targets to build each server named "dsconnect" "dsgame" and "dssearch" but will build all 3 if none are specified.

Setting Up the Database

If you do not know your mysql root password, the default on most systems is blank. It is a good idea to change this for security reasons, as your mysql root user has full permissions on all databases. The first time you run the mysql client, you should be prompted to change the password. After doing so, you will want to create a user specifically for dealing with the darkstar server database. Run mysql as follows:

mysql -u root -p

Enter your root password at the prompt. Next, create a new user as follows:

CREATE USER 'darkstar'@'localhost' IDENTIFIED BY 'password';

Substitute "password" with a password of your choice. Next, we want to create a database, and set up our new user with permissions to work with it.

CREATE DATABASE dspdb;
USE dspdb;
GRANT ALL PRIVILEGES ON dspdb.* TO 'darkstar'@'localhost';
exit

This will make it to where your new user can do anything necessary, but can only connect from localhost, so even if your password is compromised, it will be difficult to mess with the database remotely. If you need to be able to access the database remotely (like if the database is stored on a separate machine from the server), change the configuration according to the host that will be accessing the database.

Now, you need to populate the database.

cd sql
for f in *.sql
  do
     echo -n "Importing $f into the database..."
     mysql dspdb -u darkstar -ppassword < $f && echo "Success"      
  done
cd ..

To set up your ZoneIP, see the related windows section Building_the_Server#Configure_the_ZoneIP_for_the_Map_Server above for how to calculate it, and then run:

mysql -u darkstar -ppassword
USE dspdb;
UPDATE zone_settings SET zoneip = '127.0.0.1';
exit

replacing "127.0.0.1" with the value you determined in your calculation. You should see a message that at least 256 changes have occured after the update command.

Configuring the Server

Before running your new servers, you will need to update the configuration files to tell it how to talk to the database. You will need to update the files in the conf/ sub-folder named "login_darkstar.conf" "map_darkstar.conf" and "search_server.conf" at a minimum. Find the mysql settings in each and change as appropriate, using the new user, password and database you created earlier.

You may also wish to browse through the various configuration files for any settings that might be of interest to you, but more details on server customization can be found at Customizing Your Server.

Starting the Servers

You may now start your new servers. If you are using a desktop environment, you may wish to open 3 separate terminals and run each server in a separate one to monitor them efficiently. Otherwise, you will want to run them in screen sessions as follows:

screen -d -m -S dsconnect ./dsconnect
screen -d -m -S dsgame ./dsgame
screen -d -m -S dssearch ./dssearch

This will start and auto-detach all three servers, assigning them meaningful names to make it easy to reattach later by using the commands:

screen -r dsconnect

or

screen -r dsgame

or

screen -r dssearch

to monitor a given server. Once you've attached to a screen, you can detach by first pressing "ctrl+a" and then "d". There are many other useful ways to use screen, so do whatever suits yoru preference.

Your server is now up-and-running, and a client should be able to connect as long as there are no obvious errors on any of the server screens. If the search server is having issues, it is not vital for basic game functionality, but you will be unable to use search functions or the auction house. If you have run into any major issues, contact bluekirby0 (the current maintainer of the linux port as of the time of this writing) on Discord or in the forums.

Automatic Server Restarts & Crash Logs

If your server goes down, no one can connect. If it goes down when you're not looking, it can be down for an extended period of time. To fix this, you probably want to have a way to have your server auto-restart. In addition, it's good to know when the server crashed.

Getting the software

First, you'll want to download Knas Restarter here: [2].

Creating the Logging files

In the dsp directory where your game, connect, and search server reside, create a 'tools' folder. In the folder create 4 new text files by right clicking, and going to New > Text Document

Make sure you have file extension view turned on so that you can see the .txt at the end.

Rename the 4 files to the following:

connect_crash.bat <Change the file type to bat file, deleting .txt

connect_crash_log.txt

game_crash.bat <Change the file type to bat file, deleting .txt

game_crash_log.txt

Right click on connect_crash.bat and click 'edit' Place the following inside: echo %date% %time% >> connect_crash_log.txt Save and close

Right click on game_crash.bat and click 'edit' Place the following inside: echo %date% %time% >> game_crash_log.txt Save and close

Connect Server Restart

Launch Knas Restarter Find DSConnect-server.exe and click on it Click "Select" Change itnerval to 10 seconds Under "Restart Paramters"; Check Keep restarter minimized in "Working Directory" enter the directory where dsp is installed Under Actions; Check "External Script" and browse to your tools folder and select connect_crash.bat Click "Create Shortcut" and save to your desktop

Game Server Restart

Launch Knas Restarter Find DSGame-server.exe and click on it Click "Select" Change itnerval to 10 seconds Under "Restart Paramters"; Check Keep restarter minimized in "Working Directory" enter the directory where dsp is installed Under Actions; Check "External Script" and browse to your tools folder and select game_crash.bat Click "Create Shortcut" and save to your desktop

Launching

Open the shortcuts you created on your desktop, and open the search server as you normally would.

Crash logs will be found in the text files you created in the tools directory showing the date and time the crash occurred.

Problems

In some Windows opperating systems, a window will pop up saying the program has crashed, leaving the crashed program up and preventing an auto-restart. This can be avoided by disabling the user interface for critical errors. To do this follow these steps:

Start > Run gpedit.msc

Computer Configuration > Administrative Templates > Windows Components > Windows Error Reporting Set Prevent Display of the user interface for critical errors" to Enabled

Common Errors

Getting Errors on Load / In Game

You have to run all changed sql files! Be careful you keep zoneip correct in zone_settings.sql.

Can't Connect to Game Server

Make sure your zoneip in zone_settings table is correct. Also make sure packets can reach the 53445 port.

Restart server after changing zoneip.

Packet or key is invalid:

Restart login server, game server and game client.

make: *** No rule to make target

Delete .deps

source autogen.sh
./configure
make

MySQL Error 1067 Invalid default value for 'timecreate' or whatever time related

mysql -uUSER -pPASS -hlocalhost dspdb -e "set sql_mode = '';"

and try again

Credits

The original windows guide was written by thrydwolf, and has been formatted for the wiki and updated by bluekirby0