Oct 18

Denyhosts on CentOS

In general, I install fail2ban and denyhosts on all of my external linux servers that have port 22 open.  This is generally only because sftp is also installed on these systems because marketing people don’t know any other options like S3 on AWS.

I want to point out the files that need to be looked at before you enable the deamon mode via: sudo service denyhosts start.

First is the config located at /etc/denyhosts.conf.  That tells it to look at your /var/log/secure and update the /etc/host.deny file among other things

If you have IP’s or hosts that need to be whitelisted, you need to add them to a file that belongs to denyhosts.  It’s at: /var/lib/denyhosts/allowed-hosts

Once you start the service it will list all of the hosts that will be denied, verify that the list doesn’t include anything that matters to you.  If it does then you need to stop the service and delete those entries from /etc/hosts.deny and add them to /var/lib/denyhosts/allowed-hosts

Then enable it at startup with: sudo chkconfig denyhosts on

And if you on systemctl then take a look here for startup: http://digitalsos.com/?p=45


Mar 22

netdom join Error 53, Error 2, DNS is correct. Windows 2012 R2 doesn’t like /ou

We have been using a script for years that will join a Windows system to our domain.  Now with 2012 R2 it never executed.  And this was not a DNS issue.  Ping your DC’s with the friendly name from the system first and if they resolve you are good.

At first I thought was an issue with our 2012 R2 domain controllers.  Apparently after researching this I saw that a duplicate SPN check can cause this.  https://support.microsoft.com/en-us/kb/3070083

Patch your DC’s this hotfix was incorporated into later patches so if your up-to-date then you should be fine.

Remove the /ou section of your netdom join statement.  It worked for us for years, but now it just throws a error every time.  And before you say anything I tried using CN for the computers section most of the time.  Our statement was:

netdom join $serverName /d:$Domain /ou:”OU=Computers,DC=cloud,DC=digitalsos,DC=com” /ud:SOS\joinUS /pd:$decrypted /reboot:20 >> $logfile

Now it’s just:

netdom join $serverName /d:$Domain /ud:joinUS /pd:$decrypted /reboot:20 >> $logfile

Hopefully you won’t throw away a week of your time chasing this down.

Feb 04

JDBC connector to MSSQL 2014

YAFD (Yet another fucking developer) that needs to use java to connect to YAFDB (Yet Another Fucking DataBase).

So start off and install Oracle java JRE 8 on your sql server.  Then go here and get the latest Microsoft JDBC Driver, 4.2 http://www.microsoft.com/en-us/download/details.aspx?id=11774

Go to your control Panel –> System and Security –> System. Click on the left side “Advanced system settings”.  Click Environmental Variables.  Click on NEW in the System variables and for the name use CLASSPATH.  And if you extracted the zip correctly you can use this for the value: C:\Program Files\Microsoft JDBC Driver 4.2 for SQL Server\sqljdbc_4.2\enu\sqljdbc42.jar


So now we need to build the connection string as outlined here: https://msdn.microsoft.com/en-us/library/ms378428%28v=sql.110%29.aspx


Feb 04

Ubuntu SNMP config for Zabbix and Checkpoint

I’m setting up SNMP monitoring for our Checkpoint devices in AWS and Zabbix needs the SNMP client configured.  A good tutorial is located here:

But adding the templates for checkpoint was more involved than I thought.  I grabbed the templates here: https://share.zabbix.com/network-appliances/checkpoint-fw-1-hardware. Then created the mapping by going to Administrator –> General –> On the far right pull down to Value Mappings, and create new.

Then added the discovery scripts with: sudo cp advsnmp.discovery /usr/lib/zabbix/externalscripts/.

Then got the checkpoint mib file and added it to /usr/share/snmp/mibs/.  As long as you commented out the mib line in /etc/snmp/snmp.conf then you should be able to run snmptranslate -m +CHECKPOINT-MIB -IR -On memFreeReal64.0 and get an accurate translation.

The real test is to snmpwalk from the Zabbix server, for SNMPv3 use the following: snmpwalk -v3 -u UserName -l authPriv -a MD5 -A UserPassword -x DES -X EncryptionPassword memActiveReal64.0

Finally, and I’m not sure if this helped but I exported the MIb via: export MIB=+CHECKPOINT-MIB

Some useful web links: http://www.net-snmp.org/wiki/index.php/TUT:Using_and_loading_MIBS

And the net-snmp FAQ is really good: http://net-snmp.sourceforge.net/docs/FAQ.html#How_do_I_add_a_MIB_to_the_tools_

Oct 30

Build bitcoind from source Fedora 22

Disclaimer – this does NOT work.  It’s close, but no love.

As usual there are no good instructions on the net to do this.

First get the source, I already had git installed and I’m actually building Feathercoin instead of bitcoin but it should be the same for both.  Also I’m presuming that you already installed the build-essentials like gcc.  If not at a minium you should have done:

sudo dnf install automake gcc-c++ openssl-devel gcc make

Go to the folder or make a new folder like bitcoin then

git clone https://github.com/FeatherCoin/Feathercoin.git

Now we need to get and compile Berkeley DB 4.8

wget http://download.oracle.com/berkeley-db/db-4.8.30.tar.gz
tar -xvzf db-4.8.30.tar.gz
Go to /build_unix/
../dist/configure --prefix=/usr/local --enable-cxx
(as root) make install

Install the boost C++ files, and the qrencode, And if you want the GUI also add the protobuf:

sudo dnf install boost-devel qrencode-devel protobuf-devel

Now you can run the standard build process thats listed under doc/build-linux.md


Jun 12

Moving AD groups from one domain to another using ldifde

This is something that I almost never do so I’m documenting it here to avoid future mistakes.

We are migrating between domains and while we considered ADMT, there was too much clutter and our AD structure changed.  So a piece meal approach was decided on.  Hint to future self, don’t do this again, use ADMT!

I’m transferring groups but this has proved problematic.  Two main reasons for this, the AD structure is different and the Domain is different.

Big lesson first up is that you need to exclude many SAM entry’s using the -o modifier.  My command for groups striped out everything but member.  However all the member mappings changed so I excluded them as well.  I ended up with:

ldifde -f exportfile.ldf -s <serverName> -d “OU=Tomcat,OU=Security Groups,OU=IBM,DC=<domain>,DC=<domain>,DC=local” -r “(objectCategory=CN=group,CN=Schema,CN=Configuration,DC=<domain>,DC=<domain>,DC=local)” -o “badPasswordTime,badPwdCount,lastLogoff,lastLogon,logonCount,memberOf,objectGUID,objectSid,primaryGroupID,pwdLastSet,sAMAccountType,member”

Now to bring that over.  I can convert using the -c option.  In my case the OU’s lined up on the other side so the command was:

ldifde -i -f exportfile.ldf -s <serverName> -k -v -c “DC=<domain>,DC=<domain>,DC=local” “DC=<domain>,DC=<domain>,DC=org”

I have all the groups but no membership.  Use ADMT next time.

Errors that you can get with any of the SAM accounts is discussed here: https://support.microsoft.com/en-us/kb/276382


May 20

Tomcat 8 redirect and force SSL

Edit the tomcat8/conf/server.xml and add the following for 80, and another for 8080 if need be.

<Connector port=”80″ protocol=”HTTP/1.1″
redirectPort=”443″ />

Now Edit the tomcat8/conf/web.xml and at the bottom just above </web-app> put in the following and changing Entire Application to your application in webapps.

<!– SSL settings. only allow HTTPS access to MY APPLICATION –>
<web-resource-name>Entire Application</web-resource-name>
<!– auth-constraint goes here if you requre authentication –>

Now restart the tomcat service.

May 19

Java 8, tomcat 8, SSL setup from pfx, using 443

This took me a day to setup on a new CentOS Amazon image. To be honest I’d never configured SSL for tomcat before, and this was the first time that I’d used tomcat8. So I just want to go over the steps I had to do so I’ll remember all of the tweeks needed.

Configuring SSL was more painful than I expected. First issue was that I had to break up the Microsoft IIS formated certificate I had. Fortunatly that I’ve done before. From novell

First create a new folder for all of this.
Type: mkdir cert Type: cd cert
Now get the Intermediate and root certificates from your CA place them in the folder.
Get the .pfx certificate and put it in the folder.

To export the private key without a passphrase or password.
Type: openssl pkcs12 -in filename.pfx -nocerts -nodes -out key.pem

To Generate a public version of the private RSAkey
Type: openssl rsa -in key.pem -out server.key

To export the Certificate
Type: openssl pkcs12 -in filename.pfx -clcerts -nokeys -out cert.pem

The directory will now have a file cert.pem and a key.pem

Now from apache.org

* key.pem – your certificate’s private key
* cert.pem – your certificate
* domainIntermediate.crt – Organization Validation intermediate
* inter.crt – the intermediate CA that signed your certificate
* root.crt – the root CA that signed the intermediate CA

First, concatenate the CA certs, make sure the intermediate CA goes first:
Note on this – For the chain ON A NORMAL LOAD BALANCER, it’s intermediate first then domain Intermediate then the root, BUT if you want a unified cert like we are doing here the order is different, it would be domain Intermediate, then CA Intermediate, then the CA Root.  Makes no sense to me but for Comodo it is so.

$ cat domainIntermediate.crt inter.crt root.crt > chain.crt

Next, export the pkcs12 file:

$ openssl pkcs12 -export -chain -inkey key.pem -in cert.pem\
-name “server” -CAfile chain.crt -out server.p12

When prompt for export password, enter something and don’t leave it empty.

Now, use keytool to verify:

$ keytool -list -v -storetype pkcs12 -keystore server.p12

Enter the export password for the keystore password. Then you should see
a line like this from the output:

Certificate chain length: 3

Tomcat8 should now be able to use that server.p12 file as it’s keystore.
Move the server.p12 to the tomcat home directory which is /usr/share/tomcat8/
Make sure tomcat is the owner, Type: chmod tomcat:tomcat server.p12
This server needs to use 443 instead of 8443. To do that we need to tweek java permissions.
I used the guide at confluence but used the 5th option:

If using Linux 2.6.24 or later, you can set up a file capability on the java executable, to give elevated privileges to allow opening privileged ports only, and no other superuser privileges:
# setcap cap_net_bind_service+ep /path/to/bin/java
After setting this you may notice errors when starting Java like this, for example:
$ java -version
/path/to/bin/java: error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory
This means that the library is being imported from a dynamic path, and not in the trusted ld.so path. See http://bugs.sun.com/view_bug.do?bug_id=7157699 for details. To fix this, you need to locate the library, and add its path to the ld.so configuration. Note that the below is an example, and this may differ depending on Linux distribution. Replace JAVA_HOME with the correct location:
$ find JAVA_HOME -name ‘libjli.so’

# echo “JAVA_HOME/lib/amd64/jli” > /etc/ld.so.conf.d/java-libjli.conf
# ldconfig -v
After setting this all up, you need to make sure that Confluence only starts java with the direct binary path, and not via a symbolic link, otherwise the capability will not be picked up.
Setting this up means that any user can open privileged ports using Java, which may or may not be acceptable for you

At this point I usually switch user to tomcat. to do that edit /etc/passwd and change tomcat user to use /bin/bash
then as root su tomcat

We need to edit /etc/tomcat8/server.xml
Add a new connector like this:
port=”443″ maxThreads=”200″
scheme=”https” secure=”true” SSLEnabled=”true”
keystoreFile=”${user.home}/server.p12″ keystoreType=”PKCS12″ keystorePass=”changeit”
clientAuth=”false” sslProtocol=”TLS”/>

Also in my case the application was to live on the root so to do that find the host section and add Context like so:

<Host appBase=”webapps” autoDeploy=”true” name=”localhost” unpackWARs=”true”>
<Context docBase=”/var/lib/tomcat8/webapps/YourAppName” path=”” reloadable=”true” />

exit out of the tomcat account change it back to nologin then restart tomcat. Easy right?

Dec 05

How to build Openssl from source

Recently we had to install openssl but for the 32-bit platform. This is how I did it.

wget http://www.openssl.org/source/openssl-0.9.8x.tar.gz
tar zxvf openssl-0.9.8x.tar.gz
setarch i386 ./config -m32 shared
make clean; make install

In general though for anything that you need to build the following command will work:
./configure && make && sudo make install

Nov 19

Chef change attributes node level Java cookbook

How do you change the attributes at the node level.  I just went through this recently with some Scala servers that required Oracle Java 7.  The defaults for the Java cookbook at the marketplace: https://supermarket.getchef.com/cookbooks/java#readme are openJDK 6.  To replace them you need to:

knife node edit <NODE_NAME>

In the editor that pops up add the java section under normal.  In the precidence order normal will overwrite the attributes in the java file.

"name": "AWSSERVER-UE1T",
"chef_environment": "TEST",
"normal": {
"java": {
"install_flavor": "oracle",
"jdk_version": "7",
"oracle": {
"accept_oracle_download_terms": true
"set_fqdn": "AWSSERVER-UE1T.aws.test.local",
"agency": "VIV",
"tags": [

"run_list": [

Now with that said after running chef-client locally on the server it didn’t pickup the changes.  But you will see the change in the json attributes on the chef server.

Amazon has a good atricle on overriding attributes here: http://docs.aws.amazon.com/opsworks/latest/userguide/workingcookbook-json-override.html

Older posts «