Search Results for: error 0

Masking data in logs and RSYSLOG

As a mobile payments company, we at SumUp are obligated to follow many industry regulations, one of them being PCI DSS. Restricting access to credit card numbers is a clear need and this implies ensuring they are not part of the logs which are used for various purposes and have bigger audience, not restricted to the authorized list of employees who have access to sensitive data.

PAN or primary account number is used by card issuers as card number which is unique and brings information about the issuer and also in majority of the cases can be validated with Luhn algorithm. This is the number on you credit or debit card and it should be kept secret by us.

One widely used approach is to have quality assurance of the logs all over the development and deployment cycle. This is a needed and valuable attitude however first it takes a lot of human resources and second it is kind of reactive approach in terms of dealing with production systems. So we want something better, something mandatory which can leave us on the safe side if we got human error somewhere in the chain. This is very important in our case where we need to put logging management system out of PCI scope. From four ways which are offered by PCI DSS, Requirement 3.4:

3.4 Render PAN unreadable anywhere it is stored (including on portable digital media, backup media, and in logs)
by using any of the following approaches:

  •  One-way hashes based on strong cryptography, (hash must be of the entire PAN).
  • Truncation (hashing cannot be used to replace the truncated segment of PAN).
  •  Index tokens and pads (pads must be securely stored).
  • Strong cryptography with associated key-management processes and procedures.

our natural choice for log messages is truncation. We want to truncate PAN data if it’s present in the logs for some reason in example in situation when temporary the log level is increased for investigation. While we have centralized log storage which is in PCI scope we want to transfer the logs  in real-time in some external location, accessible for developers and BI where they can find and use the information they need.

Since we are using rsyslog for logging daemon our next step was to get in touch with Adiscon – the company behind this brilliant piece of software. They were very interested when I explained the idea and the work started. A little bit later we got new message modification module called mmexternal. It sends the message to some external binary and expects an input. More on the implementation here.

Let me give you an example with a code snippet from rsyslog config and an example of python script which is doing a regular expression to catch and replace i.e. VISA, MasterCard and AMEX cards. You may find a lot of useful regular expressions here:

rsyslog.conf

module(load=”mmexternal”)
action(type=”mmexternal”
binary=”/usr/local/bin/external_python_cards_replace.py”
interface.input=”msg” )

external_python_cards_replace.py

Please note that the above snippets are only examples. With using regular expressions you are going to have many false positives but in general this won’t be an issue. Also note that you can modify completely different parts of the logs and also you are not limited to any language or technique for doing so.

With the following example we have negligible resource consumption on the server where log modification is done. Synthetic test which not claim for accuracy shows around 5% CPU usage on single core 2.5GHz virtual CPU for 100 messages/s.

This is how we are doing it. All comments and suggestions are welcome!

rsyslog 8.2.2 (v8-stable) released

We have just released 8.2.2 of the v8-stable branch.

This release allows to build the missing (contributed) modules under v8. Please note, that most of these modules did not yet receive real testing as we don’t have the necessary environments. We depend on users submitting error reports and helping out to iron out any issues that may arise.

Additionaly, the separated documentation is available as a tarball download on the download page.

ChangeLog:

http://www.rsyslog.com/changelog-for-8-2-2-v8-stable/

Download:

http://www.rsyslog.com/downloads/download-v8-stable/

As always, feedback is appreciated.

Best regards,
Florian Riedl

Changelog for 8.2.2 (v8-stable)

Version 8.2.2 [v8-stable] 2014-06-02

  • made the missing (contributed) modules build under v8
    Note that we could do this to the stable, because there is NO regression chance at all: only the modules themselves were changed, and they did NOT work at all previously. Please also note that most of these modules did not yet receive real testing. As we don’t have the necessary environments (easily enough available), we depend on users submitting error reports and helping to iron out any issues that may arise.
    Modules:

    • mmrfc5424addhmac
    • omrabbitmq
    • omgssapi
    • omhdfs
    • omzmq3

Changelog for 8.3.0 (v8-devel)

Version 8.3.0 [v8-devel] 2014-04-10

  • new plugin for anonymizing credit card numbers
    Thanks to Peter Slavov for providing the code.
  • external message modification modules are now supported
    They are bound via the new native module “mmexternal”. Also, a sample skeleton for an external python message modification module has been added.
  • new $jsonmesg property with JSON representation of whole message object
    closes: https://github.com/rsyslog/rsyslog/issues/19
  • improved error message for invalid field extraction in string template
    see also:
    http://kb.monitorware.com/problem-with-field-based-extraction-t12299.html
  • fix build problems on Solaris
  • NOTE: a json-c API that we begun to use requires the compiler to be in c99 mode. By default, we select it automatically. If you modify this and use gcc, be sure to include “-std=c99” in your compiler flags. This seems to be necessary only for older versions of gcc.

Changelog for 8.2.0 (v8-stable)

Version 8.2.0 [v8-stable] 2014-04-02
This starts a new stable branch based on 8.1.6 plus the following changes:

  • we now use doc from the rsyslog-doc project
    As such, the ./doc subtree has been removed. Instead, a cache of the rsyslog-doc project’s files has been included in ./rsyslog-doc.tar.gz. Note that the exact distribution mode for the doc is still under discussion and may change in future releases. This was agreed upon on the rsyslog mailing list. For doc issues and corrections, be sure to work with the rsyslog-doc project. It is currently hosted at https://github.com/rsyslog/rsyslog-doc
  • add support for specifying the liblogging-stdlog channel spec
    new global parameter “stdlog.channelspec”
  • add “defaultnetstreamdrivercertfile” global variable to set a default for the certfile.
    Thanks to Radu Gheorghe for the patch.
  • omelasticsearch: add new “usehttps” parameter for secured connections
    Thanks to Radu Gheorghe for the patch.
  • “action resumed” message now also specifies module type which makes troubleshooting a bit easier. Note that we cannot output all the config details (like destination etc) as this would require much more elaborate code changes, which we at least do not like to do in the stable version.
  • add capability to override GnuTLS path in build process
    Thanks to Clayton Shotwell for the patch
  • better and more consistent action naming, action queues now always contain the word “queue” after the action name
  • bugfix: ompipe did resume itself even when it was still in error
    See: https://github.com/rsyslog/rsyslog/issues/35
    Thanks to github user schplat for reporting

Changelog for 7.6.3 (v7-stable)

Version 7.6.3 [v7.6-stable] 2014-03-27

  • add capability to override GnuTLS path in build process
    Thanks to Clayton Shotwell for the patch
  • support for librelp 1.2.5
    Support new return states of librelp 1.2.5 to emit better error messages. For obvious reasons, librelp 1.2.5 is now required.
  • bugfix: ompipe used invalid default template
    This is a regression from an old change (didn’t track it down precisely, but over a year ago). It used the Forwarding template instead of the file template (so we have a full syslog header). This fix corrects it back to previous behaviour, but new scripts that used the wrong format may now need to have the RSYSLOG_ForwardingFormat template explicitely be applied.
    closes: https://github.com/rsyslog/rsyslog/issues/50
  • bugfix: ompipe did emit many suspension messages for /dev/xconsole
    (hopefully now) closes: https://github.com/rsyslog/rsyslog/issues/35
    When it was present, but nobody reading from it. The problem is the way the rsyslog v7 engine tries to resolve failures in outputs. It does some retries, and along those lines some state information gets lost and it is close to impossible to retain it. However, the actual root problem is that ompipe does not reliably detect if it is able to recover. The problem here is that it actually does not know this before it does an actual write. These two things together mess up the logic that suppresses invalid resumption/suspension messages (actually, the plugin switches state really that often). Nevertheless, the prime problem with /dev/xconsole (and probably most other pipes as well) is that it gets full. So I have now added code that checks, during resume processing, if the pipe is writable. If it is not, resume is deferred. That should address the case.

rsyslog 7.6.2 (v7-stable) released

This version introduces the support for librelp 1.2.4, thus it it fixes a problem with librelp in the last release.

 

Note that now librelp 1.2.4 is required as we process it’s new error codes emitted when librelp does not support TLS.

 

ChangeLog:

http://www.rsyslog.com/changelog-for-7-6-2-v7-stable/

Download:

http://www.rsyslog.com/downloads/download-v7-stable/

As always, feedback is appreciated.

Best regards,
Florian Riedl

rsyslog 7.6.1 (v7-stable) released

This version is the first bug-fixing release for 7.6. In addition to bugfixes, it also offers some new, unintrusive, features. Most importantly, the way action suspension and resumption messages can now be configured on a finger grain basis. The action naming has been improved and kept consistent and imjournal can now better handle invalid entries in systemd journal.

This releases is a recommended update for all users of the 7.6 branch.

Note that now librelp 1.2.3 is required as we process it’s new error codes emitted when librelp does not support TLS.

ChangeLog:

http://www.rsyslog.com/changelog-for-7-6-1-v7-stable/

Download:

http://www.rsyslog.com/rsyslog-7-6-1-v7-stable/

As always, feedback is appreciated.

Best regards,
Florian Riedl

Changelog for 7.6.1 (v7-stable)

Version 7.6.1 [v7.6-stable] 2014-03-13

  • added “action.reportSuspension” action parameter This now permits to control handling on a per-action basis rather to the previous “global setting only”.
  • “action resumed” message now also specifies module type which makes troubleshooting a bit easier. Note that we cannot output all the config details (like destination etc) as this would require much more elaborate code changes, which we at least do not like to do in the stable version.
  • better and more consistent action naming, action queues now always contain the word “queue” after the action name – add support for “tls-less” librelp we now require librelp 1.2.3, as we need the new error code definition See also: https://github.com/rsyslog/librelp/issues/1
  • build system improvements
    • autoconf subdir option
    • support for newer json-c packages Thanks to Michael Biebl for the patches.
  • imjournal enhancements:
    • log entries with empty message field are no longer ignored
    • invalid facility and severity values are replaced by defaults
    • new config parameters to set default facility and severity Thanks to Tomas Heinrich for implementing this
  • bugfix: ompipe did resume itself even when it was still in error See: https://github.com/rsyslog/rsyslog/issues/35 Thanks to github user schplat for reporting
  • bugfix: “action xxx suspended” did report incorrect error code
  • bugfix: ommongodb’s template parameter was mandatory but should have been optional Thanks to Alain for the analysis and the patch.
  • bugfix: only partial doc was put into distribution tarball Thanks to Michael Biebl for alerting us. see also: https://github.com/rsyslog/rsyslog/issues/31
  • bugfix: async ruleset did process already-deleted messages Thanks to John Novotny for the patch.

librelp 1.2.3

librelp 1.2.3 [download]

This version of librelp addresses the problem that RHEL and CENTOS provide too-old versions of GnuTLS to make support librelp’s TLS functionality. To “solve” this problem, TLS can now be disabled in librelp, so that base RELP functionality is also available on those platforms.

Note that librelp previously built on those platforms, but DID NOT WORK.

Version 1.2.3 – 2014-03-13
– add ability to build librelp without TLS
This is required on some platforms where GnuTLS is too old
Use –disable-tls to select this mode. If set, RELP_RET_ERR_NO_TLS
will be returned on calls that selected TLS mode.
See also: https://github.com/rsyslog/librelp/issues/1
– API change: two functions that used to return void now return state
these are:
* relpSrvEnableTLS();
* relpSrvEnableTLSZip();
This change is acceptable because the state must not necessary be
evaluated. If not, the same error is returned a bit later in the
calling sequence. Having it early, however, may help with better
error messages. So it’s still optional to check.
– bugfix: configure required too-old version of GnuTLS

  This lead to successful builds which later on would not execute.

sha256sum: c06e15f94f7e75d7c9f8c66faefaf5ab8808072c828ac6982c70825fe3aa20f1

Scroll to top