Ubuntu: Ignoring Transitive Trust Domains when using Samba/Winbind

ubuntuIf your Ubuntu host is authenticating against an Active Directory Domain Controller, you may find there are multiple subdomains or transitive trusts visible.  Which is not a problem in most cases – but if your host is in a subnet where a connection to these other subdomain or transitive trust domains is not possible, you can experience long delays until a timeout period is reached by the SMB client.

To get a list of all the visible domains, including transitive trusts:

wbinfo -m

Continue reading “Ubuntu: Ignoring Transitive Trust Domains when using Samba/Winbind”

OpenWrt: Enabling HTTPS for the LuCI Web Admin Interface

openwrt_logoBy default, LuCI, the web admin interface for OpenWrt is not HTTPS enabled.  This may not be a critical issue for you since it is a LAN facing service, but the type of infrastructure information being exchanged combined with the fact that it is usually accessed over WiFi protocols might make you want to consider it – especially considering it is a 5 minute fix.

First connect to OpenWrt either via ssh with Dropbear, or via the USB-TTL cable and a terminal program.  Install the following packages:

opkg update
opkg install luci-lib-px5g
opkg install px5g-standalone
opkg install libustream-openssl

Continue reading “OpenWrt: Enabling HTTPS for the LuCI Web Admin Interface”

OpenWrt: Flashing Linksys WRT1X00AC/S from USB-TTL Using Ubuntu

openwrt_logoFlashing the firmware of the Linksys WRT1X00AC/S is well documented on the OpenWrt wiki.  So I don’t feel the need to go over the architectural concepts in this article, but I did want to provide instructions for the Ubuntu specific tools you can use to flash the firmware.

If you want to try flashing to OpenWrt using the factory LinkSys ‘Router Firmware Update’ feature, that is your choice, but it really is working blind and you have no ability to fix issues if something goes wrong.  After bricking my router once, I now rely solely on the Serial to USB-TTL cable which is the highly recommended connectivity method from the OpenWrt page.

Step 1. Connect via USB-TTL cable

I wrote a detailed article about using the Adafruit USB TTL Serial cable to connect to the Linksys WRT1X00AC/S for an Ubuntu host.

After powering off/on the router, you should be able to clearly the see the boot sequence of your Linksys firmware in your terminal program.  Below is a snippet of the output showing the Linksys logo in ASCII art which scrolls by as the router brings up all its services.

linksys_factor_booting2

Continue reading “OpenWrt: Flashing Linksys WRT1X00AC/S from USB-TTL Using Ubuntu”

OpenWrt: Installing LuCI Web Interface after Deploying latest OpenWrt Image

openwrt_logoThe stable OpenWrt images are built with LuCI, an OpenWrt web administration interface.  But if you are using the bleeding edge or trunk OpenWrt images, then you won’t get this package.

Luckily, it is not difficult to add the LuCI package to the install.  As long as you have Dropbear enabled for ssh access, or you are connected via UBS-TTL and have shell access to your router then it only takes a few commands.

opkg update
opkg install luci
/etc/init.d/uhttpd enable

Continue reading “OpenWrt: Installing LuCI Web Interface after Deploying latest OpenWrt Image”

Ubuntu: Enabling the Ubuntu universe Repository

ubuntuThere are four main repositories for Ubuntu: Main, Universe, Restricted, and Multiverse.  The Ubuntu CD contains the packages from the Main and Restricted repositories, so even if you do not have an Internet connections those will be available.

However, if you have booted from the LiveCD, and did not initially configure a wired or wireless network connection, then the ‘Universe’ repository will not be enabled.

If you were trying to install a package such as putty and the Universe repository source was disabled, you would get ‘E: Unable to locate package’ responses when trying to install and an empty response from apt-cache when searching for this package:

Continue reading “Ubuntu: Enabling the Ubuntu universe Repository”

OpenWrt: Installing a TFTP Server on Ubuntu for OpenWrt Firmware Updates

openwrt_logoThe Trivial File Transfer Protocol (TFTP) is an extremely simple protocol most often used for network booting strategies, such as PXE and flashing OpenWrt images unto consumer routers.

I go over full instructions for flashing OpenWrt using Ubuntu and flashing a sysupgrade in another post, this article will focus specifically on setting up a tftp server daemon on Ubuntu that can be used to serve the binary image file.

Installation

First, install the tftp server and client packages:

# apt-get install tftpd-hpa tftp-hpa -y

Continue reading “OpenWrt: Installing a TFTP Server on Ubuntu for OpenWrt Firmware Updates”

SaltStack: Validating States of Minion without Execution

saltstack_logo-thumbnailBefore running state.apply against a minion, especially in a production environment, a good sanity test can be to list the states that will be executed without actually running those states.

This can be done by adding tests=True to the end of the state command. For example, to check all the states that will be applied to a minion:

salt 'myminion' state.apply tests=True

Or to check which states would be run for the apache formula:

salt 'myminion' state.sls apache tests=True

 

GIT: Calling git Clone Using Password with Special Character

gitlogoIt is more popular to use an ssh key instead of a password when automating a git clone from a guest OS.  But if you do need to specify the password directly into the console command, it takes this form:

$ git clone https://<user>:<password>@<gitserver>/<path>/<repo>.git

Which works fine if the password is plaintext, but if it has special characters like an exclamation mark, you need to use percent encoding which is often called URL encoding.

Continue reading “GIT: Calling git Clone Using Password with Special Character”

Ubuntu: Hang While Installing gutenprint as Network Driver

If you experience hanging when installing the gutenprint drivers for a network printer from the desktop, try manually installing the gutenprint drivers from the console first.

Most likely, you will see a screen like below, and the progress bar will continually cycle but never end.

gutenprint-searching2

If you can’t cancel, you can use the ‘xkill’ command from the console and click on the dialog window.  But you will also need to kill the process, and that can be done by  finding the process id using:

Continue reading “Ubuntu: Hang While Installing gutenprint as Network Driver”

Ubuntu: Installing Packages without Public Internet Access

ubuntuIn production data centers, it is not uncommon to have limited public internet access due to security policies.  So while running ‘apt-get’ or adding a repository to sources.list is easy in your development lab, you have to figure out an alternative installation strategy because you need a process that looks the same across both development and production.

For some, building containers or images will satisfy this requirement.  The container/image can be built once in development, and transferred as an immutable entity to production.

But for those that use automated configuration management such as Salt/Chef/Ansible/Puppet to layer components on top of a base image inside a restricted environment, there is a need to get binary packages to these guest OS without requiring public internet access.

There are several approaches that could be taken: using an offline repository or a tool such as Synaptic or Keryx or apt-mirror, but in this post I’ll go over using apt-get on an internet connected source machine to download the  necessary packages for Apache2, and then running dpkg on the non-connected target machine to install each required .deb package and get a running instance of Apache2.

Note that this solution only addresses the apt packages.  If you need to pull down Javascript packages from npm or Python modules from pypi,  then you might want to look at my article on using a squid proxy to whitelist specific URL.

Continue reading “Ubuntu: Installing Packages without Public Internet Access”

Ubuntu: Extending a virtualized disk when using LVM

ubuntuIt is common for a virtualized Guest OS base image to have a generic minimal storage capacity.  But this capacity can easily be exceeded by production scenarios, performance testing, logging, or even the general cruft of running a machine 24×7.

In a previous post, I described extending a virtualized disk when using classic partitions.  In this post, I will perform the same task but with an LVM enabled system.  We will use console level tools so that it could be done from a remote terminal or by automation.

Continue reading “Ubuntu: Extending a virtualized disk when using LVM”

Ubuntu: Creating a Samba/CIFS share to quickly share files with Windows

ubuntuWe live in a multi-platform world, and the ability to easily share folders of content between users in the same protected network is a function made very convenient in the Windows world with CIFS shares (e.g. \\mydesktop\sharedfolder).

Luckily for Ubuntu users, it is pretty easy to setup CIFS shares to offer that same interoperability with Windows hosts on your network.  Start by installing the Samba components.

apt-get install samba -y

Continue reading “Ubuntu: Creating a Samba/CIFS share to quickly share files with Windows”

vRealize Log Insight: Creating your own content pack for field extraction

vmware_logo Content Packs are plugins that allow you you to create pre-packaged knowledge about specific event types.

For example, you can create a content pack that knows how to extract fields from one of your custom log sources.  Beyond extracted fields, you can also add saved queries, aggregations, alerts, dashboards, and visualizations.

Incoming Events from Agent

First, let’s examine our sample log file on the agent side, in a file named /tmp/test.log.

2016-07-14 22:04:13.233 INFO  com.my.myTest      - [  150] 200

Continue reading “vRealize Log Insight: Creating your own content pack for field extraction”

OpenWrt: Use setenv firmwareName for newer versions of Linksys WRT1900AC/S

openwrt_logoWhen flashing an OpenWrt image to your newer versioned WRT1900AC/S, be aware that instead of using ‘setenv firmware_name’, you should instead use ‘setenv firmwareName’.

The command will not fail, but the router will not understand that it should look for a non-default name for the image and your tftp transfer will fail.

This change appears to have been made between WRT1900AC V1 and WRT1900AC V2.  So, for the latest versions such as WRT1900ACS, be sure to use ‘setenv firmwareName’.

Ubuntu: Serial level access to your Linksys WRT1X00AC/S

ubuntuWhether you are updating the official LinkSys router firmware or taking it a step further and installing open-source firware like OpenWrt, serial level access to your Linksys router is the most dependable way of guaranteeing a connection.

And if you have tried to flash the firmware via the web admin interface and after a reboot you cannot get web access again, then you have no choice.  You have to be able to plug directly into the router’s serial interface and troubleshoot.

Continue reading “Ubuntu: Serial level access to your Linksys WRT1X00AC/S”

Ubuntu: Extending a virtualized disk using fdisk when not using LVM

ubuntuIt is common for a virtualized Guest OS base image to have a generic minimal storage capacity.  But this capacity can easily be exceeded by production scenarios, performance testing, logging, or even the general cruft of running a machine 24×7.

For this reason, extending a virtualized disk can be extremely helpful.  Here is a walk through for extending a disk using fdisk on an Ubuntu system that is using classic partitions.  For performing this operation with LVM enabled, see my post here.

This type of change is typically made with a live CD to ensure exclusive disk access and gparted GUI for convenience.  But we will use fdisk here so that it could be done from a remote terminal or by automation.

Continue reading “Ubuntu: Extending a virtualized disk using fdisk when not using LVM”

Logstash: Using metrics to debug the filtering process

elastic-logstash-fw When building your logstash filter, you would often like to validate your assumptions on a large sampling of input events without sending all the output to ElasticSearch.

Using Logstash metrics and conditionals, we can easily show:

  • How many input events were processed successfully
  • How many input events had errors
  • An error file containing each event that processed in error

This technique gives you the ability to track your success rate across a large input set, and then do a postmortem review of each event that failed.

I’ll walk you through a Logstash conf file that illustrates this concept.

Continue reading “Logstash: Using metrics to debug the filtering process”

Ubuntu: Using a swap file instead of swap partition for virtualized server VMs

ubuntuBefore virtualization, there was a stronger argument for using a swap partition instead of a swap file for servers.  A fragmented swap file could lead to performance issues that a statically sized and placed partition did not have consider.

But once virtualization comes into play, unless you go to great lengths to segment your storage pools, that swap partition is not guaranteed to be either statically sized or statically placed on a physical platter.  And at that point, you should consider using a swap file which provides more flexibility in sizing and capacity planning.

Here are instructions for adding a 16Gb swap file to Ubuntu:

Continue reading “Ubuntu: Using a swap file instead of swap partition for virtualized server VMs”

Ubuntu: Using pdftk to stitch together two-sided PDF

ubuntuThere are many consumer side printers that provide the ability to scan a document to PDF.  But unless you have a high-end series, the printer may only be capable of scanning one side at a time, which means you end up with a “front.pdf” and “back.pdf”.

If you have a Linux desktop or laptop, luckily the solution is as simple as calling ‘pdftk’.

Continue reading “Ubuntu: Using pdftk to stitch together two-sided PDF”

Logstash: Testing Logstash grok patterns online

elastic-logstash-fwIn my previous posts, I have shown how to test grok patterns locally using Ruby on Linux and Windows.  This works well when your VM do not have full internet access, or only have console access, or any reason that you want to test it locally.

If you have access to a graphical web browser and the log file, there is a nice online grok constructor here and here. and by simply entering a sampling of the log lines and a grok pattern, you can verify that all the lines are parsed correctly.

Here is a small example to start you off:

Continue reading “Logstash: Testing Logstash grok patterns online”

Logstash: Testing Logstash grok patterns locally on Windows

elastic-logstash-fwIf the logs you are shipping to Logstash are from a Windows OS, it makes it even more difficult to quickly troubleshoot a grok pattern being sent to the Logstash service.

It can be beneficial to quickly validate your grok patterns directly on the Windows host.  Here is an easy way to test a log against a grok pattern:

Continue reading “Logstash: Testing Logstash grok patterns locally on Windows”

Documentum: JMS Access Logs to Analyze Custom Method Load

LogoDocumentumA vital piece of information that often goes overlooked is the load created by standard and custom methods run on the Java Method Server.  In some applications (such as D2), the JMS is used extensively for application functionality and this can have performance implications to your end users.

You can capture this information by enabling the JMS access log, which is not enabled by default. Continue reading “Documentum: JMS Access Logs to Analyze Custom Method Load”

Documentum: Separating dfc.properties from your WAR

LogoDocumentumIn the world of microservices and containers, it is often desirable to keep settings such as those found in dfc.properties outside of the jar or war so that the deployment binary is the same no matter which environment it is deployed into.

The settings in dfc.properties can be externalized by specifying the location of dfc.properties in a JVM system property such as:

-Ddfc.properties.file=/tmp/dfc.properties

Sending SMTP Mail from Windows Using PowerShell

When working from the Windows command line, you can do a quick test to validate your SMTP connectivity using PowerShell:

 

c:\> Powershell -executionpolicy bypass

PS c:\> Send-MailMessage –to <TO> –from <FROM> –subject "testing123" –body "this is a test" –smtpserver <SMTPServer> -port 25

And if the mail server is accessed over TLS/SSL with SMTP authentication enabled:

PS c:\> Send-MailMessage –to <TO> –from <FROM> –subject "testing456" –body "this is a secure test" –smtpserver <SMTPServer> -port 587 -UseSsl -Credential (Get-Credential)

This is easier than going down to telnet, which is typically not installed on a modern Windows host: Continue reading “Sending SMTP Mail from Windows Using PowerShell”

Documentum: Ignoring Referrals from the LDAP Synch Job

icon-ldapThe most common way of integrating your existing Identity Management system with Documentum is to offer SSO (Single Sign-On) via the LDAP Synchronization job.

This requires that you set a Base DN for Documentum to search through, but it is not uncommon when dealing with real-world LDAP servers to have LDAP referrals in that search space. This is transparent, but it can cause performance issues, and even cause the job to timeout if the forwarded DNS name is not resolvable from the Content Server host.

Continue reading “Documentum: Ignoring Referrals from the LDAP Synch Job”

EMC OnDemand: Federated Identity Management and Silent SSO

Identity Management for On-Premise Applications

Our industry today has some very proven technologies for providing a single set of login credentials to applications installed on-premise.  Most commonly, companies use a central Identity Management system (e.g. Microsoft Active Directory/Oracle Internet Directory/IBM Tivoli), and these systems implement an LDAP interface that 3rd party applications can call to validate user credentials.

This allows end users to login to their internal HR portal, SharePoint site, or local Documentum Webtop with the same credentials they used to gain entrance into their Windows Desktop, and is termed SSO (Single Sign-On).  This has dramatically improved the end user experience, as well as improved the ability of IT to mange the risk and policies surrounding identity management.

Continue reading “EMC OnDemand: Federated Identity Management and Silent SSO”

EMC OnDemand: Best Practices for Custom Methods

The concept of custom methods which run directly on the Java Method Server has proven an extremely useful extension point for Documentum developers and solutions architects.  Whether used in a workflow activity to integrate with an enterprise message queue or as an action for Webtop users who need temporarily escalated privileges to apply legal retention, custom Java methods have become a key customization in most customer environments. Features include:

  • Lightweight invocation of methods as compared to dmbasic and external Java methods that require execution
  • DFC operations execute on the same host as the Content Server which minimizes the effects of network latency and throughput
  • Can be configured to run as the repository owner which allows them elevated privileges to content when necessary
  • Provide the logic for workflow auto-activities, able to utilize any Java library including the DFC
  • Provide the logic for custom job/methods, again able to utilize the full power of Java and its libraries

Continue reading “EMC OnDemand: Best Practices for Custom Methods”