Bundesbrandschatzamt
Bring Your SQL To The Next Level
When you start learning SQL and databases it is trivial to find good resources. My adventures started with ORACLE8i The Complete Reference. Tutorials are another way. With the official documentation for your RDBMS on hand you can learn the specific dialect of SQL you need. Later in your career you might cross the way of an SQL guru. In that point in time you wonder: How can I reach that level? I met quite Read more…
Howto Write Complex Programs Running On Vertica
Even though my roots are in the field of system administration I learned early on how important SQL and Databases are. Over the years I worked on different Projects on Oracle, MySQL and Vertica. Both from a DBA as well as a developer perspective. Even though the examples here are for Vertica most of it can and should be adapted to other databases as well.
If you have to process and aggregate data there are different ways to tackle the problem. I have seen a lot of programs fetching the complete data out of the Database trying to do the aggregation within the application and FAIL. Even in areas where they should have known better Read more…
It’s Good to be Lazy
Tips for note taking for system administrators and SREs I wrote over at USENIX ;login.
GPG
GNU Privacy Guard (GnuPG or GPG) was first released in 1999. Over the years it found a wide usage not only limited to exchanging encrypted emails. I want to give here a brief introduction how to Read more…
Perl Fork
One of the reasons why i still stick with Perl are the database modules around DBI. I just like the beauty of the syntax. Many years ago i learned that you can bring processes to speed with process forking. Even though i used the fork function every now and then i wasn’t concerned much about the childs outcome. If something failed the next run of the program would take care of it. The child processes Read more…
Amanda backup and AWS S3
After I had my DDS Streamer at home running for a while again with AMANDA, the Advanced Maryland Automatic Network Disk Archiver, I was considering using it at work as well. Always looking at cost optimization our Vertica clusters looked like good candidates. As most DBAs I fear loosing data. Following the safe route I use vbr hardcopy backup followed by an rsync to a dedicated backup EBS volume. From those we take Read more…
Amanda backup
Many moons ago I had to convince my former employer that we should have backup. There are only 2 kinds of companies: those without backup and those loosing data. wait. those are the same. Aside from lack of backup we were in the need of more storage. Finally I got backup and learned a bit about Fibreoptic, SAN storage and tapelibraries. Of Read more…
Emacs and SQL
In the last year I had the pleasure of rewriting a data calculation software originally written in Java in Vertica SQL. As you can imagine I am not talking about simple select statements. A lot of try and error was involved. For debugging you would run those queries manually in an interactive SQL session. That way you can dive Read more…
Teaching Linux Part 3
After a few sessions around Emacs I started to continue with Linux.
Every process on an operating system must belong to somebody to define
which privileges this process can have. Which means a very important
task of the OS is the user management.
In Linux the first entry point is /etc/nsswitch.conf provided by the
Read more…
Teaching Linux Part 2
Finally I managed to get my sessions back into our calendar. As a lot of vacation is going on it turned out to be a single session. Even if you can teach a single person it is well spend time. Plus nobody get’s shy asking questions. As the old saying says “there are no silly questions” some people still think they exist. Read more…
Teaching Linux Part 1
Recently we got 2 student interns at work. Times have changed since I started working and I decided it would be a good opportunity to pass on some of my knowledge.
I decided to start at zero. In my oppinion this is always a good Read more…
Vertica and Emacs
If you have paid attention to my other blog entries you have already figured that I am a fan of Literate Database Work as well as a user of Vertica. sql-mode and sql-interactive-mode are quite powerful. I combined them with yasnippet. The product is vertica-snippets - a package you can install via MELPA. If you are like me you probably
Blogging via Orgmode
For the last 6 years I used Perl and the Catalyst MVC framework for my homepage. It provides a lot of nice features. The only issue: I haven’t used a lot of them. Being a heavy Emacs user and to follow the KISS principle I wanted to switch to Orgmode and org-publish. Thanks to Howard Abrams this turned out quite easy. I copied some of his elisp code to give my own Emacs the capabilities. As I liked his Webdesign I analyzed and transformed it into my own - of course asking for permission down the road. Finally I managed to update my collection of links and invested the extra time to redirect URLs from my previous website.
With a much easier way to publish I can just hope I am going to spend more time in the future writing about more topics.
Vertica backup and restore
disclaimer: use at your own risk
If you pay close attentation to the HP Vertica documentation you will see this requirement for doing a restore:
The cluster to which you are restoring the backup has the same number of hosts as the one used to create the backup. The node names and the IP addresses must also be identical.
UPDATE: Since Vertica 9.2 most of this information is opsolete. See Restoring Objects to an Alternate Cluster.
500 commits
Having 500 git commits on an editors configuration files is quite a number. I think it’s a good reason for another recap.
A machinist builds his own tools. The same rule should apply to somebody working in Development or Systems Administration.
It makes your live easier and in the end it will speed you up.
Org Tables
No production change on Friday rule and git
As a sysadmin you probably know the rule: No production changes on Friday! If you use git to manage your automation and happen to not using git tags to finally send your code life this little code snippet might be for you:
Emacs Keyboard Map
Recently I found an old print of a Emacs keyboard map. As it is quite useful I searched the internet to find the source for this print. I figured the old saying upload something to the internet and it’s saved forever isn’t true. Searching my ancient notebooks I was lucky and found the pdf file. I couldn’t reach the original Author Sami BEN GRINE but found the original announcement post at https://groups.google.com/forum/m/#!topic/gnu.emacs.help/t2XkXn-0UJ4:
Literate DevOps
Even as on advocate of automation I see the need of taking notes of your systems. Legacy systems and stuff that cannot be automated for whatever reason. Using a logbook in plain text is a good choice.
If you call this file LOGBOOK.org and use Emacs you can save a lot
of hassle. Besides beeing a good editor Emacs is capable of running
shells. org-mode gives you a markup language and the capability of
executing code blocks in that shell.
Literate Database work with Oracle
Literate Database Work by Howard Abrams was eye opening.
My only problem: I am currently in the Oracle world and orgmode ob-sql doesn’t support sqlplus. With SQLi-Mode it’s at least possible to use Emacs to connect to a database. In this case Oracle. But ob-sql supports Perl dbi-shell. For Perl there is DBD::Oracle which brings us full circle.
Emacs into the beyond
In the last 20 years I used a lot of programs but Emacs is still around. The one thing I regret: I should have started earlier diving into it. It’s 18 month since I started using git for my Emacs configuration. 180 commits - time for a resume.
Tip table
bbabookmarks
How to share bookmarks within your department or company?
Maybe you even didn’t thought about this solution. Often wikis are used for this task. You have to visit an extra website to get to your final destination. Quite cumbersome if you ask me.
Emacs
I’ve been using EMACS for the better part of my sysadmin live. Back in the days I’ve learned the basics of editing. Every now and then I learned another shortcut or command.
Things changed with my current position of a Senior Sysadmin: Now I spend most of the day in Emacs. Writing hundreds of lines of code and configfiles I figured it was time to dig deeper into Emacs to save time and make my life easier.
partitioning
A couple of month ago we had a discussion about partitioning on the USENIX/Sage mailing list. Laziness tends to configure only 3 partitions today: /boot, swap and /. With LVM you can easily resize partitions and there are quite some advantages:
Booting DVD from a USB stick
Just a couple of days ago it happened again: A DVD sitting on my desk and a computer ready for installation without a DVD drive. The only difference this time: I had no USB DVD drive.
port based routing
Lets assume you are using LinuxVirtualServer in direct routing mode.
Now you are in the situation that your cluster nodes have to talk to the outside world to fetch rss feeds or talk to authentication systems. These systems are not under your control and might change the ip address and you wouldn’t get informed.
webcaching
Operating a globally accessed webservice for the last couple of years traffic costs, web performance and user experience are familiar topics for me.
All these topics come down to web optimization and caching.
The first and probably easiest task is activating gzip compression. There are some known issues with eg some version of Internet Explorer. In my setup i made good experience with this line for Apache:
IPMI
I am using IPMI and other out-of-band management interfaces since 2006.
Sometimes they are referred to as BMC, ILOM or Remote Supervisor Adapter (RSA). There are quite some differences:
Cheapest models supply only features like power on/off, reset and serial-over-lan. They work quite well and fit most needs. If you spend more money you get features like upload of floppy images, mounting remote CD/DVD or even KVM capabilities. Nowadays you have often a web interface. Sometimes you get a dedicated LAN port for it. Quite often its shared with LAN0. Both versions work well and it depends more on your grade of paranoia which type you prefer. If you buy server hardware you get the basic module for free. In a Linux environment you can stay with that type. For Windows I recommend using the KVM version. Even though Windows Server can do serial-over-lan since version 2003.
git
Even though i have to admit that git has a high learning curve it’s worth the time.
I started with the ProGIT book and use it today for a wide range:
- source code
- to manage the images of my camera
- in conjunction with etckeeper
