Categories
Database

Search for lost data

A simplified description of a workflow in one of our supported application is:

AnotherServer is sending data through a middleware to MyAppServer. We store the received data in a MS-SQL database an process the data.

Now there is a problem in the AnotherServer system and they did not know, which data they have send to MyAppServer. So they send us a list of data IDs from that day and asked us to check in our application, which data was not received or received but not processed.

The list contains 600 IDs and it would take ~ 1 minute to check a single ID. So this would be work for ~10 hours. And not one single hour would be fun. After a short discussion we decided to do the data analytics directly in the MyApp database.

List of values

First problem is, that they send us an excel file containing the data IDs and we have to transform this in a way, so we can use this IDs in a SQL statement:

SELECT TempTable.Field1
FROM ( VALUES
(10),
(11),
(12)
      ) AS TempTable (Field1)

Connect list to table

LEFT OUTER JOIN [MyAppSchema].[ReceivedDataTable]
ON TempTable.Field1 = ReceivedDataTable.Data_ID

GROUP BY Field1, Data_ID, is_Processed
ORDER BY is_Processed DESC

Filter processed data

WHERE is_Processed != 1 
   OR is_Processed IS NULL
Categories
Java

Remote Debugging

I need to directly debug on the application Tomcat server, not only on my local Tomcat instance.

Compile Java code

For debugging we need to keep the line numbers while compiling.
To build the war file we use an ANT script and we have to add the debug and debuglevel attributes in the javac tag:

<javac srcdir="${src}" destdir="${build}" 
       includeantruntime="false" 
       encoding="UTF-8" source="1.8" 
       fork="true" 
       debug="on" debuglevel="lines,vars,source" 
       verbose="true">
  [...]
</javac>

Configure Tomcat server

To enable remote debugging on the Tomcat server add some arguments to CATALINA_OPTS in setenv.sh file:

ATALINA_HOME=/app/myApp/tomcat
CATALINA_BASE=/app/myApp/tomcat
CATALINA_PID=/app/myApp/tomcat/tomcat.pid
JAVA_HOME=/app/java/jdk8u265-b01-jre
CATALINA_OPTS="$CATALINA_OPTS -Djava.library.path=/app/library -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n"
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/app/library
export LD_LIBRARY_PATH
JAVA_OPTS="${JAVA_OPTS} -Djavax.net.ssl.trustStore=/app/certs/corporate_truststore.jks -Djavax.net.ssl.trustStorePassword=secret -Xms256M -Xmx1024M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/app/myApp/dump"

Configure Eclipse

Create a new debug configuration:


Categories
Linux

NFS

NFS - Network File System

We have two Linux servers, each with one Tomcat application server. Both application servers must have access to one folder to up- and download files.

We will create a folder on the first server and enable access through the network on this server for the second server.

Operating System: RedHat Enterprise Linux 7

myAppServer1 - Server

vim /etc/exports
 
systemctl list-units *nfs-server*
# if 0 loaded units, you have to enable and start
 
systemctl enable nfs-server.service
systemctl start nfs-server.service
systemctl list-units *nfs-server*
exportfs -r
exportfs -s
/app/myApp/uploads myAppServer2.mynetwork.net(rw,sync)

myAppServer2 - Client

vim /etc/fstab
vim /etc/auto.master
vim /etc/auto.misc
 
systemctl enable autofs.service
systemctl start autofs.service
 
su tomcat
ls -lisah /app/myApp/uploads
# add:
myAppServer1.mynetwork.net:/app/myApp/uploads   /app/myApp/uploads     nfs     user,timeo=3,bg 0       0
# add:
/misc   /etc/auto.misc  --timeout 0 --negative-timeout 5
# add:
myapp_uploads     -fstype=nfs,bg,timeout=-1,timeo=5,rw,tcp,soft,intr,nosuid myAppServer1.mynetwork.net:/app/myApp/uploads

Test

Reboot both servers and doublecheck that the folder is still available and accessible from both servers.

reboot now
 
su tomcat
ls -lisah /app/myApp/uploads

Open webapplication in your browser on myAppServer1 and upload a file. Test download of the file.
Switch to myAppServer2 and download file.

Open webapplication in your browser on myAppServer2 and upload a file. Test download of the file.
Switch to myAppServer1 and download file.

NFS4 - Update

Please also note this post: NFS4

Categories
Java

Deploy to Tomcat with Maven

In my last post I decribed the way to build and deploy a war file to Tomcat application server with Ant.

In this post I show how to deploy the build war file to Tomcat application server with Maven.

Create inside the "ant" folder this Maven file:

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>deringo</groupId>
    <artifactId>myApp</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>war</packaging>
    <name>myApp</name>
    <description>myApp Maven deployment</description>

    <properties>
        <java.version>1.8</java.version>
        <tomcat.version>8.5.33</tomcat.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.tomcat</groupId>
            <artifactId>tomcat-catalina</artifactId>
            <version>${tomcat.version}</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.0</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>3.2.2</version>
                <configuration>
                    <failOnMissingWebXml>false</failOnMissingWebXml>
                    <warName>myApp</warName>
                    <wtpContextName>myApp</wtpContextName>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.tomcat.maven</groupId>
                <artifactId>tomcat7-maven-plugin</artifactId>
                <version>2.2</version>
                <configuration>
                    <url>http://myAppServer:7011/manager/text</url>
                    <username>tomcat</username>
                    <password>tomcat</password>
                    <warFile>dist/myApp.war</warFile>
                    <path>/myApp</path>
                    <update>true</update>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

Execute with goal: tomcat7:deploy

Use Properties file

In my ANT file I have used a properties file for tomcat username/password etc. In my Maven script I want also to use this properties file.

Unfortunatly Maven can not handle property files out of the box. But there is a Plugin we can use:

        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>properties-maven-plugin</artifactId>
            <version>1.0-alpha-1</version>
            <executions>
                <execution>
                    <phase>initialize</phase>
                    <goals>
                        <goal>read-project-properties</goal>
                    </goals>
                    <configuration>
                        <files>
                            <file>tomcat.properties</file>
                        </files>
                    </configuration>
                </execution>
            </executions>
        </plugin>

This is the same, as writing this directly into pom.xml:

<properties>
    <tomcat.username>tomcat</tomcat.username>
    <tomcat.password>tomcat</tomcat.password>
</properties>

The complete pom.xml with properties:

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>deringo</groupId>
  <artifactId>myApp</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>war</packaging>
  <name>myApp</name>
  <description>myApp Maven deployment</description>

  <properties>
    <java.version>1.8</java.version>
    <tomcat.version>8.5.33</tomcat.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.apache.tomcat</groupId>
      <artifactId>tomcat-catalina</artifactId>
      <version>${tomcat.version}</version>
      <scope>provided</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>properties-maven-plugin</artifactId>
        <version>1.0-alpha-1</version>
        <executions>
          <execution>
            <phase>initialize</phase>
            <goals>
              <goal>read-project-properties</goal>
            </goals>
            <configuration>
              <files>
                <file>tomcat.properties</file>
              </files>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.0</version>
        <configuration>
          <source>${java.version}</source>
          <target>${java.version}</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>3.2.2</version>
        <configuration>
          <failOnMissingWebXml>false</failOnMissingWebXml>
          <warName>myApp</warName>
          <wtpContextName>myApp</wtpContextName>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.tomcat.maven</groupId>
        <artifactId>tomcat7-maven-plugin</artifactId>
        <version>2.2</version>
        <configuration>
          <url>${tomcat.manager.url}</url>
          <username>${tomcat.username}</username>
          <password>${tomcat.password}</password>
          <warFile>dist/myApp.war</warFile>
          <path>${webapp.name}</path>
          <update>true</update>
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>

Update 2021-03-24

I tried the configuration in a project but got this error:

Plugin execution not covered by lifecycle configuration: org.codehaus.mojo:properties-maven-plugin:1.0-alpha-1:read-project-properties (execution: default, phase: initialize)

I solved the error adding a pluginManagement tag:

<build>
    <pluginManagement>
        <plugins>
            <plugin> ... </plugin>
            <plugin> ... </plugin>
                  ....
        </plugins>
    </pluginManagement>
</build>

But this solution seems more like a workaround when reading the discussion on stackoverflow.
...need to search further...

Use in Jenkins

I have not tried it, but it should be possible to set an environment variable in Jenkins build step to execute with a configurable filename.

For example TOMCAT_PROPERTIES=jenkins-home/secret/tomcat-dev.properties.
And we can use it in Maven script this way:

            <configuration>
              <files>
                <file>${env.TOMCAT_PROPERTIES}</file>
              </files>
            </configuration>
Categories
Java

Deploy to Tomcat

Deploy myApp.war file to Tomcat application server, using Tomcat manager app and an ant script.

Make a folder "ant" in your project and this it the folder for all other actions below.

Make a folder "dist" and generate your myApp.war file into this folder with the "build.xml" script.

Make a folder "tomcat-libs" and copy following files from your tomcat installation:

  • catalina-ant.jar
  • catalina.jar
  • servlet-api.jar
  • jsp-api.jar

Create file "tomcat.properties" with properties for tomcat server and tomcat manager app.

Create "deploy.xml" file for deployment.

Execute deploy.xml script with ant, it must show the info sections with properties of your tomcat.properties file.

Execute deploy-webapp target from deploy.xml script to deploy your myApp.war file to tomcat server through the tomcat manager app.

If you are using for example Jenkins, you can set the "secprops.location" from outside the script, so you can use the same script for different tomcat installations.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE project>
<project name="myApp" default="info" basedir=".">

  <description>
    myApp ant deploy file.
  </description>

  <property name="secprops.location" value="tomcat.properties" />
  <property file="${secprops.location}" />

  <property name="war.file" value="dist/myApp.war" />

  <target name="info">
    <echo message="-------------------------------------------------------------------" />
    <echo message="Commands:" />
    <echo message="  start-webapp" />
    <echo message="  stop-webapp" />
    <echo message="  undeploy-webapp" />
    <echo message="  deploy-webapp" />
    <echo message="  sessions-webapp" />
    <echo message="-------------------------------------------------------------------" />
    <echo message="Properties: "/>
    <echo message="tomcat.manager.url = ${tomcat.manager.url}"/>
    <echo message="tomcat.username    = ${tomcat.username}"/>
    <echo message="tomcat.password    = ${tomcat.password}"/>
    <echo message="webapp.name        = ${webapp.name}"/>
    <echo message="war.file           = ${war.file}"/>
    <echo message="-------------------------------------------------------------------" />
  </target>

  <path id="catalina-ant-classpath">
    <fileset dir="tomcat-libs">
      <include name="catalina-ant.jar" />
      <include name="catalina.jar" />
    </fileset>
  </path>

  <taskdef name="catalina-start" classname="org.apache.catalina.ant.StartTask" classpathref="catalina-ant-classpath" />
  <taskdef name="catalina-stop" classname="org.apache.catalina.ant.StopTask" classpathref="catalina-ant-classpath" />
  <taskdef name="catalina-deploy" classname="org.apache.catalina.ant.DeployTask" classpathref="catalina-ant-classpath" />
  <taskdef name="catalina-undeploy" classname="org.apache.catalina.ant.UndeployTask" classpathref="catalina-ant-classpath" />
  <taskdef name="catalina-sessions" classname="org.apache.catalina.ant.SessionsTask" classpathref="catalina-ant-classpath" />
  
  <target name="sessions-webapp">
    <catalina-sessions url="${tomcat.manager.url}" username="${tomcat.username}" password="${tomcat.password}" path="/${webapp.name}" failonerror="false" />
  </target>
  
  
  <target name="stop-webapp">
    <catalina-stop url="${tomcat.manager.url}" username="${tomcat.username}" password="${tomcat.password}" path="/${webapp.name}" failonerror="false" />
  </target>

  <target name="start-webapp">
    <catalina-start url="${tomcat.manager.url}" username="${tomcat.username}" password="${tomcat.password}" path="/${webapp.name}" />
  </target>

  <target name="undeploy-webapp">
    <catalina-undeploy url="${tomcat.manager.url}" username="${tomcat.username}" password="${tomcat.password}" path="/${webapp.name}" failonerror="false" />
  </target>

  <target name="deploy-webapp">
    <echo message="START to deploy file ${war.file} to Tomcat Server: ${tomcat.manager.url}" />
    <catalina-deploy url="${tomcat.manager.url}" username="${tomcat.username}" password="${tomcat.password}" path="/${webapp.name}" war="file:${war.file}" update="true"/>
    <echo message="END of deployment" />
  </target>

</project>
tomcat.manager.url=http://myAppServer:7011/manager/text
tomcat.username=tomcat
tomcat.password=tomcat
webapp.name=myApp
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE project>
<project name="myApp" default="clean" basedir=".">
  <description>
    myApp ant build file.
  </description>
  <!-- Set global properties for this build -->
  <property name="src" location="../source"/>
  <property name="webcontent" location="../WebContent"/>
  <property name="build" location="build"/>

  <tstamp prefix="build-info">
    <format property="current-date" pattern="d-MMMM-yyyy" locale="en" />
    <format property="current-time" pattern="hh:mm:ss a z" locale="en" />
  </tstamp>

  <target name="init">
    <!-- Create the build directory structure used by compile -->
    <mkdir dir="${build}"/>
    <!-- Create the distribution directory -->
    <mkdir dir="${dist}/lib"/>
  </target>

  <target name="compile" depends="init" description="compile the source">
    <!-- Compile the Java code from ${src} into ${build} -->
    <javac srcdir="${src}" destdir="${build}" includeantruntime="false" encoding="UTF-8">
      <classpath>
        <!-- Taken from Apache Tomcat 8.5 lib folder -->
        <pathelement path="tomcat-libs/servlet-api.jar"/>
        <pathelement path="tomcat-libs/jsp-api.jar"/>
      </classpath>
    </javac>
    <!-- Copy all non-java ressoures from source folder -->
    <copy todir="${build}">
      <fileset dir="${src}" excludes="**/*.java" />
    </copy>

  </target>

  <target name="war" depends="compile">
    <war destfile="dist/myApp.war" webxml="${webcontent}/WEB-INF/web.xml">
      <fileset dir="${webcontent}"/>
      <classes dir="${build}"/>
      <manifest>
        <attribute name="Manifest-Version" value="1.0"/>
        <attribute name="Built-On" value="${build-info.current-date}"/>
        <attribute name="Built-At" value="${build-info.current-time}"/>
      </manifest>
    </war>
  </target>

  <target name="clean" description="clean up">
    <!-- Delete the ${build} directory trees -->
    <delete dir="${build}"/>
  </target>
</project>
Categories
Java Linux

Setup Tomcat manager app

In my last post I set up a Tomcat application server in general, now I enable Tomcat manager app for deployment.

# Tomcat Users
mv /app/myApp/tomcat/conf/tomcat-users.xml /app/myApp/tomcat/conf/tomcat-users.xml_original
vim /app/myApp/tomcat/conf/tomcat-users.xml

vim/app/myApp/tomcat/conf/server.xml

# By default the Manager is only accessible from a browser running on the same machine as Tomcat. If you wish to modify this restriction, you'll need to edit the Manager's context.xml file.
vim /app/myApp/tomcat/webapps/manager/META-INF/context.xml
<?xml version="1.0" encoding="UTF-8"?>
<tomcat-users xmlns="http://tomcat.apache.org/xml"
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="http://tomcat.apache.org/xml tomcat-users.xsd"
              version="1.0">
 
  <role rolename="manager-gui"/>
  <role rolename="manager-script"/>
  <user username="tomcat" password="tomcat" roles="manager-gui,manager-script"/>
 
</tomcat-users>
<Server port="7010" shutdown="SHUTDOWN">
  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>
 
  <Service name="Catalina">
    <Connector port="7011" />
   
    <!-- Define an AJP 1.3 Connector on port 7012 -->
    <Connector port="7012" protocol="AJP/1.3" secretRequired="false" />
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="myApp-dev">
      <Host name="localhost" appBase="webapps" />
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>
    </Engine>
  </Service>
</Server>
<Context antiResourceLocking="false" privileged="true" >
    <!-- ## uncomment this Block ##
    <Valve className="org.apache.catalina.valves.RemoteAddrValve"
         allow="127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1" />
    -->
</Context>

Test Tomcat manager app

Open http://myAppServer:7011/manager and login with username: tomcat and password: tomcat.

Categories
Java Linux

Setup Tomcat application server

Setup files

su myUser
 
# Java
cd /app/java
tar -xzf /app/files/OpenJDK8U-jre_x64_linux_hotspot_8u265b01.tar.gz
tar -xzf /app/files/OpenJDK11U-jre_x64_linux_hotspot_11.0.8_10.tar.gz
tar -xzf /app/files/OpenJDK15U-jre_x64_linux_hotspot_15_36.tar.gz
 
# myApp Tomcat
cd /app/myApp
tar -xzf /app/files/apache-tomcat-8.5.59.tar.gz
tar -xzf /app/files/apache-tomcat-9.0.39.tar.gz
 
# Certificate
cp /app/files/corporate_truststore.jks /app/certs/
 
# SAP JCO
cp /app/files/_sapjco3-64/3.0.19/linuxx86/libsapjco3.so /app/library/
cp /app/files/_sapjco3-64/3.0.19/linuxx86/sapjco3.jar /app/library/

Setup Tomcat

su myUser
 
# Symlink to actual Tomcat version
ln -s /app/ccp/apache-tomcat-8.5.59 /app/myApp/tomcat
 
# remove sample application
# but keep the Tomcat Manager app for deployment
rm -rf /app/myApp/tomcat/webapps/docs
rm -rf /app/myApp/tomcat/webapps/examples
rm -rf /app/myApp/tomcat/webapps/ROOT
 
# configure Tomcat
vim /app/myApp/tomcat/bin/setenv.sh
mv /app/myApp/tomcat/conf/server.xml /app/myApp/tomcat/conf/server.xml_original
vim /app/myApp/tomcat/conf/server.xml
 
# expand Classpath
vim /app/myApp/tomcat/conf/catalina.properties
# tomcat/bin/setenv.sh
CATALINA_HOME=/app/myApp/tomcat
CATALINA_BASE=/app/myApp/tomcat
CATALINA_PID=/app/myApp/tomcat/tomcat.pid
JAVA_HOME=/app/java/jdk8u265-b01-jre
CATALINA_OPTS="$CATALINA_OPTS -Djava.library.path=/app/library"
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/app/library
export LD_LIBRARY_PATH
JAVA_OPTS="${JAVA_OPTS} -Djavax.net.ssl.trustStore=/app/certs/corporate_truststore.jks -Djavax.net.ssl.trustStorePassword=notchangeit -Xms512M -Xmx2048M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/app/myApp/dump"
# tomcat/conf/catalina.properties
common.loader= [...] ,"/app/library/sapjco3.jar"
# server.xml
<Server port="7010" shutdown="SHUTDOWN">
  <Service name="Catalina">
    <Connector port="7011" />
   
    <!-- Define an AJP 1.3 Connector on port 7012 -->
    <Connector port="7012" protocol="AJP/1.3" secretRequired="false" />
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="myApp-dev">
      <Host name="localhost" appBase="webapps" />
    </Engine>
  </Service>
</Server>

Tomcat as a Service

# As root
vim /etc/systemd/system/tomcat.service
# enable script:
systemctl enable tomcat.service
# tomcat.service

# Systemd unit file for myApp tomcat
#
# To create clones of this service:
   
# Systemd unit file for tomcat
 [Unit]
 Description=myApp Tomcat Web Application Container
 After=syslog.target network.target
   
   
 [Service]
 Type=forking
   
 ExecStart=/app/myApp/tomcat/bin/startup.sh
 ExecStop=/app/myApp/tomcat/bin/shutdown.sh
   
 User=myUser
 Group=myUser
   
 [Install]
 WantedBy=multi-user.target

Service control

Enable user myUser to control Tomcat services:

visudo -f /etc/sudoers
##################################################
## Allow user myUser to control (apache & tomcat) services
%myUser ALL=(root) NOPASSWD: /bin/systemctl
%myUser ALL=(root) NOPASSWD: /usr/sbin/service

Test Tomcat

For Tomcat testing I use my ShowHeaders app (GitHub).
ShowHeaders is a minimalistic webapp that is not much more than a "Hello World", but it shows the HTTP headers, what is quite useful when testing reverse proxy integration.

# copy ShowHeaders App for Testing (also for Reverse Proxy Configuration Testing)
[myUser@DEV ~]$ cp /app/files/ShowHeaders/ROOT.war /app/myApp/tomcat/webapps/
# start tomcat:
[myUser@DEV ~]$ sudo systemctl start tomcat

#
curl localhost:7011

Test connection from outside the server itself: http://myAppServer:7011/

Categories
Linux

Copy files with rsync

I have to setup and configure a DEV server and afterwards the QA/staging and PROD servers.

I will download required software with my laptop and upload to DEV server into folder /app/files. For other servers I just want to copy this folder from the DEV server.

Setup server

DEV

# install software
yum install rsync -y

# create user
adduser myUser

# create folder
mkdir /app/files
chown myUser:myUser /app/files

# switch to myUser
su myUser

# Create Private & Public keys
ssh-keygen -t rsa -b 4096            # generates id_rsa & id_rsa.pub in  /home/myUser/.ssh/

# authorize yourself
cat /home/myUser/.ssh/id_rsa.pub >> /home/myUser/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys

QA / others

nearly the same setup, but we copy the keys from the DEV user, as he is the same as on the other servers.

# install software
yum install rsync -y

# create user
adduser myUser

# create folder
mkdir /app/files
chown myUser:myUser /app/files

# switch to myUser
su myUser

# create .ssh folder
mkdir ~/.ssh
chmod 0700 ~/.ssh

# create files, copy content from DEV server!
vi ~/.ssh/id_rsa
chmod 0600 ~/.ssh/id_rsa
vi ~/.ssh/id_rsa.pub
chmod 0600 ~/.ssh/id_rsa.pub
vi ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys

Test SSH login

# login from DEV to QA
[myUser@DEV] ssh QA
# login from QA to DEV
[myUser@QA] ssh DEV

# loggin with explicit port
[myUser@DEV] ssh QA -p 22

Download software

Copy all files to /app/files

Java

Open URL https://adoptopenjdk.net/

Click on "Andere Plattformen"

Select for each:

  • JVM: HotSpot
  • Operating System: Linux
  • Architecture: x64

Java Versions:

  • OpenJDK 8 (LTS)
  • OpenJDK 11 (LTS)
  • OpenJDK 15 (Latest)

Download the JRE files.

Tomcat

Open URL: http://tomcat.apache.org/

Download Binary Distribution → Core → tar.gz-Version of:

  • Tomcat 8.5
  • Tomcat 9

SAP JCO

Copy the latest version from the corporate repository.

Path: _sapjco3-64/versions/3.0.19

Corporate Root CA

Copy the password ("notchangeit") protected truststore: corporate_truststore.jks

Copy from server to server

[myUser@QA ~]$ rsync --progress -avhe 'ssh -p 22' myUser@DEV:/app/files/ /app/files
Categories
Linux

Logical Volume Creation

In one of my current projects I have to attach a (virtual) disk to a (cloud) server.

Gather information

To get an overview of the system I used this commands:

pvs
pvdisplay

vgs
vgdisplay

lvs
lvdisplay

fdisk -c -l /dev/sdb

Interpretation of information

Physical VolumePartitionVolume GroupLogical VolumesSize
/dev/sda107 GB
/dev/sda/dev/sda10,5 GB
/dev/sda/dev/sda2106,5 GB
/dev/sda/dev/sda2vg00<99.50g
/dev/sda/dev/sda2vg00lv_home2 GB
/dev/sda/dev/sda2vg00lv_usr2 GB
/dev/sda/dev/sda2vg00"free"<76.50g
/dev/sdb107 GB

Technically it is possible to create a new logical volume inside volume group vg00, because there are > 76,5 GB unassigned and free storage.

But the vg00 is reserved for the Operating System and should not used for data from application.

Therefore the additional disk is needed for applicatin data.

Volume creation

Physical Volume with ~ 100 GB is attached to the server and added as /dev/sdb.

A volume group "vg_app" will be created with two logical volumes. One logical volume for logfiles, named "lv_logs" with 20 GB and the other with remaining free space, named "lv_app", for application data.

# create volume group
vgcreate -s 32M vg_app /dev/sdb

# create logical volume for logfiles
lvcreate -L 20G -n lv_log vg_app

# create logical volume for application data
lvcreate -l 100%VG -n lv_app vg_app

# show information
vgdisplay -v /dev/vg_app

mount -a
Physical VolumeVolume GroupLogical VolumeSize
/dev/sdb107 GB
/dev/sdbvg_app107 GB
/dev/sdbvg_applv_log20 GB
/dev/sdbvg_applv_app> 79 GB

Create File system

mkfs -t ext4 /dev/vg_app/lv_app
mkfs -t ext4 /dev/vg_app/lv_log

Mount logical volumes

mkdir /app
mkdir /app/log
 
mount /dev/mapper/vg_app-lv_log /app/log
mount /dev/mapper/vg_app-lv_app /app
 
mount -a
vi /etc/fstab
mount -a
# /etc/fstab
# [...] existing files code [...]
/dev/mapper/vg_app-lv_app /app     ext4 defaults 0 0
/dev/mapper/vg_app-lv_log /app/log ext4 defaults 0 0

Test

Reboot and doublecheck that mounted volumes are still there.

reboot now
##########
df -h
Filesystem                 Size  Used Avail Use% Mounted on
[...]
/dev/mapper/vg_app-lv_app   79G   57M   75G   1% /app
/dev/mapper/vg_app-lv_log   20G   45M   19G   1% /app/log

-> Both logical volumes are mounted --> Success!

Categories
Java

Git & SSL

In my first post about Git I wrote about the problem with non-public CA signed certitificates and how to handle it. I did not mention the easierst (and unsecured) way to handle this, so I write this post to have all possibilities in one place.

Add certificate to truststore

I download the public certificate of the CA from webbrowser and add it to the truststore of Git.

Where is the cert store of git?

git config --system --list
http.sslcainfo={PathToGit}/mingw64/ssl/certs/ca-bundle.crt

To add the non-public CA cert to Git cert store just open ca-bundle.crt and the downloaded certificate with an text editor and copy the content of the certificate to the ca-bundle.

Use Windows Networking Layer

I configured the sslBackend to the Windows Networking Layer:

# use SChannel, the built-in Windows networking layer. This means that it will use the Windows certificate storage mechanism and you do not need to explicitly configure the curl CA storage mechanism.
git config --global http.sslBackend schannel

Disable SSL Verify

The easierst and unsecure way is to simply disable SSL validation:

git config --global http.sslVerify false

This also works with the system configuration (--system instead of --global). I did this an a project with a very short time budget, we had to configure Git on the Linux system and this Git installation was used by a Jenkins. Both servers, Git & Jenkins, are in the same corporate intranet.