Subscribe: mikas blog
http://michael-prokop.at/blog/feed/atom/
Added By: Feedage Forager Feedage Grade B rated
Language: German
Tags:
access  ansible  debian stretch  debian  format access  new  packages  release  return code  sha sum  ssh  stretch  utils 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: mikas blog

mikas blog



... and even if no one reads it



Updated: 2018-03-21T08:02:27Z

 



Event: Infracoders-Graz – Best Practices in der IT-Administration

2018-03-21T08:02:27Z

Am Dienstag (20.03.2018) findet das nächste Treffen der Infracoders-Graz statt. Ich wurde eingeladen einen Vortrag zu halten und werde zum Thema “Best Practices in der IT-Administration” referieren. Was: Vortrag zu “Best Practices in der IT-Administration” Wann: Dienstag, 20.03.2018 um 19:00 Uhr Wo: Aula X Space, Georgigasse 85, Graz Eintritt frei

Am Dienstag (20.03.2018) findet das nächste Treffen der Infracoders-Graz statt. Ich wurde eingeladen einen Vortrag zu halten und werde zum Thema “Best Practices in der IT-Administration” referieren.

  • Was: Vortrag zu “Best Practices in der IT-Administration”
  • Wann: Dienstag, 20.03.2018 um 19:00 Uhr
  • Wo: Aula X Space, Georgigasse 85, Graz
  • Eintritt frei



Revisiting 2017

2018-01-04T23:22:00Z

Mainly to recall what happened last year and to give thoughts and planning for upcoming year(s) I’m once again revisiting the last year (previous years: 2016, 2015, 2014, 2013 + 2012). Here we go: Events: none (not even Grazer Linuxdays), on purpose Technology / Open Source: lots of work around Proxmox + Ceph finally released […]

Mainly to recall what happened last year and to give thoughts and planning for upcoming year(s) I’m once again revisiting the last year (previous years: 2016, 2015, 2014, 2013 + 2012). Here we go:

Events:

Technology / Open Source:

Music:

  • Played the drums less often than I wish I did

Sports:

Business:

Personal:

  • Read ~one book per month on average, which is below my targets but better than in previous years (the non-IT books I recall are “Open: An Autobiography” by Andre Agassi, “Tiere für Fortgeschrittene” by Eva Menasse, “Du musst dich nicht entscheiden wenn du tausend Träume hast” by Barbaba Sher, “Fettnäpfchenführer Taiwan” by Deike Lautenschläger, “Der kleine Prinz” by Antoine De Saint-Exu, “Gebrauchsanweisung für Spanien” by Paul Ingendaay, “Irmgard Griss: im Gespräch mit Carina Kerschbaumer” by Carina Kerschbaumer, “Die letzte Ausfahrt” by Markus Huber)
  • Continued with taking care of my kids every Monday and half of Tuesday (which is still challenging every now and then with running your own business, but it’s so incredibly important and worth the effort)
  • Started to learn Spanish (maintaining a 354 day streak on Duolingo until the end of 2017)

Conclusion: after a very challenging 2016 I took several arrangements to ensure a better 2017. Recording further metrics about my daily work helped me with capacity and workload planning. I tested lots of different paper notebooks and workflows to improve my daily routines and work, including The Five Minute Journal and the Pomodoro Technique. Trying to get enough sleep and avoid work in the evenings/nights as much as possible (overall it became an exception and not the norm) improved my work-life-balance. In 2018 I’ll get back to attending a few selected events (incl. Fosdem, Grazer Linuxdays and DebConf) and work on some new projects. Exciting times ahead, looking forward to 2018!




Usage of Ansible for Continuous Configuration Management

2017-12-25T08:57:02Z

It all started with a tweet of mine: I received quite some feedback since then and I’d like to iterate on this. I’m a puppet user since ~2008 and since ~2015 also ansible is part of my sysadmin toolbox. Recently certain ansible setups I’m involved in grew faster than I’d like to see, both in […]It all started with a tweet of mine: I received quite some feedback since then and I’d like to iterate on this. I’m a puppet user since ~2008 and since ~2015 also ansible is part of my sysadmin toolbox. Recently certain ansible setups I’m involved in grew faster than I’d like to see, both in terms of managed hosts/services as well as the size of the ansible playbooks. I like ansible for ad hoc tasks, like `ansible -i ansible_hosts all -m shell -a 'lsb_release -rs'` to get an overview what distribution release systems are running, requiring only a working SSH connection and python on the client systems. ansible-cmdb provides a nice and simple to use ad hoc host overview without much effort and overhead. I even have puppetdb_to_ansible scripts to query a puppetdb via its API and generate host lists for usage with ansible on-the-fly. Ansible certainly has its use case for e.g. bootstrapping systems, orchestration and handling deployments. Ansible has an easier learning curve than e.g. puppet and this might seem to be the underlying reason for its usage for tasks it’s not really good at. To be more precise: IMO ansible is a bad choice for continuous configuration management. Some observations, though YMMV: ansible’s vaults are no real replacement for something like puppet’s hiera (though Jerakia might mitigate at least the pain regarding data lookups) ansible runs are slow, and get slower with every single task you add having a push model with ansible instead of pull (like puppet’s agent mode) implies you don’t get/force regular runs all the time, and your ansible playbooks might just not work anymore once you (have to) touch them again the lack of a DSL results in e.g. each single package management having its own module (apt, dnf, yum,….), having too many ways how to do something, resulting more often than not in something I’d tend to call spaghetti code the lack of community modules comparable to Puppet’s Forge the lack of a central DB (like puppetdb) means you can’t do something like with puppet’s exported resources, which is useful e.g. for central ssh hostkey handling, monitoring checks,… the lack of a resources DAG in ansible might look like a welcome simplification in the beginning, but its absence is becoming a problem when complexity and requirements grow (example: delete all unmanaged files from a directory) it’s not easy at all to have ansible run automated and remotely on a couple of hundred hosts without stumbling over anything — Rudolph Bott as complexity grows, the limitations of Ansible’s (lack of a) language become more maddening — Felix Frank Let me be clear: I’m in no way saying that puppet doesn’t have its problems (side-rant: it took way too long until Debian/stretch was properly supported by puppets’ AIO packages). I had and still have all my ups and downs with it, though in 2017 and especially since puppet v5 it works fine enough for all my use cases at a diverse set of customers. Whenever I can choose between puppet and ansible for continuous configuration management (without having any host specific restrictions like unsupported architectures, memory limitations,… that puppet wouldn’t properly support) I prefer puppet. Ansible can and does exist as a nice addition next to puppet for me, even if MCollective/Choria is available. Ansible has its use cases, just not for continuous configuration management for me. The hardest part is to leave some tool behind once you reached the end of its scale. Once you feel like a tool takes more effort than it is worth you should take a step back a[...]



Grml 2017.05 – Codename Freedatensuppe

2017-06-14T20:46:19Z

The Debian stretch release is going to happen soon (on 2017-06-17) and since our latest Grml release is based on a very recent version of Debian stretch I’m taking this as opportunity to announce it also here. So by the end of May we released a new stable release of Grml (the Debian based live […]

The Debian stretch release is going to happen soon (on 2017-06-17) and since our latest Grml release is based on a very recent version of Debian stretch I’m taking this as opportunity to announce it also here. So by the end of May we released a new stable release of Grml (the Debian based live system focusing on system administrator’s needs), known as version 2017.05 with codename Freedatensuppe.

Details about the changes of the new release are available in the official release notes and as usual the ISOs are available via grml.org/download.

With this new Grml release we finally made the switch from file-rc to systemd. From a user’s point of view this doesn’t change that much, though to prevent having to answer even more mails regarding the switch I wrote down some thoughts in Grml’s FAQ. There are some things that we still need to improve and sort out, but overall the switch to systemd so far went better than anticipated (thanks a lot to the pkg-systemd folks, especially Felipe Sateler and Michael Biebl!).

And last but not least, Darshaka Pathirana helped me a lot with the systemd integration and polishing the release, many thanks!

Happy Grml-ing!




The #newinstretch game: dbgsym packages in Debian/stretch

2017-05-25T09:31:38Z

Debug packages include debug symbols and so far were usually named -dbg in Debian. Those packages are essential if you’ve to debug failing (especially: crashing) programs. Since December 2015 Debian has automatic dbgsym packages, being built by default. Those packages are available as -dbgsym, so starting with Debian/stretch you should no longer look for -dbg […]Debug packages include debug symbols and so far were usually named -dbg in Debian. Those packages are essential if you’ve to debug failing (especially: crashing) programs. Since December 2015 Debian has automatic dbgsym packages, being built by default. Those packages are available as -dbgsym, so starting with Debian/stretch you should no longer look for -dbg packages but for -dbgsym instead. Currently there are 13.369 dbgsym packages available for the amd64 architecture of Debian/stretch, comparing this to the 2.250 packages which I counted being available for Debian/jessie this is really a huge improvement. (If you’re interested in the details of dbgsym packages as a package maintainer take a look at the Automatic Debug Packages page in the Debian wiki.) The dbgsym packages are NOT provided by the usual Debian archive though (which is good thing, since those packages are quite disk space consuming, e.g. just the amd64 stretch mirror of debian-debug consumes 47GB). Instead there’s a new archive called debian-debug. To get access to the dbgsym packages via the debian-debug suite on your Debian/stretch system include the following entry in your apt’s sources.list configuration (replace deb.debian.org with whatever mirror you prefer): deb http://deb.debian.org/debian-debug/ stretch-debug main If you’re not yet familiar with usage of such debug packages let me give you a short demo. Let’s start with sending SIGILL (Illegal Instruction) to a running sha256sum process, causing it to generate a so called core dump file: % sha256sum /dev/urandom & [1] 1126 % kill -4 1126 % [1]+ Illegal instruction (core dumped) sha256sum /dev/urandom % ls core $ file core core: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'sha256sum /dev/urandom', real uid: 1000, effective uid: 1000, real gid: 1000, effective gid: 1000, execfn: '/usr/bin/sha256sum', platform: 'x86_64' Now we can run the GNU Debugger (gdb) on this core file, executing: % gdb sha256sum core [...] Type "apropos word" to search for commands related to "word"... Reading symbols from sha256sum...(no debugging symbols found)...done. [New LWP 1126] Core was generated by `sha256sum /dev/urandom'. Program terminated with signal SIGILL, Illegal instruction. #0 0x000055fe9aab63db in ?? () (gdb) bt #0 0x000055fe9aab63db in ?? () #1 0x000055fe9aab8606 in ?? () #2 0x000055fe9aab4e5b in ?? () #3 0x000055fe9aab42ea in ?? () #4 0x00007faec30872b1 in __libc_start_main (main=0x55fe9aab3ae0, argc=2, argv=0x7ffc512951f8, init=, fini=, rtld_fini=, stack_end=0x7ffc512951e8) at ../csu/libc-start.c:291 #5 0x000055fe9aab4b5a in ?? () (gdb) As you can see by the several “??” question marks, the “bt” command (short for backtrace) doesn’t provide useful information. So let’s install the according debug package, which is coreutils-dbgsym in this case (since the sha256sum binary which generated the core file is part of the coreutils package). Then let’s rerun the same gdb steps: % gdb sha256sum core [...] Type "apropos word" to search for commands related to "word"... Reading symbols from sha256sum...Reading symbols from /usr/lib/debug/.build-id/a4/b946ef7c161f2d215518ca38d3f0300bcbdbb7.debug...done. done. [New LWP 1126] Core was generated by `sha256sum /dev/urandom'. Program terminated with signal SIGILL, Illegal instruction. #0 0x000055fe9aab6[...]



The #newinstretch game: new forensic packages in Debian/stretch

2017-05-25T07:48:50Z

Repeating what I did for the last Debian releases with the #newinwheezy and #newinjessie games it’s time for the #newinstretch game: Debian/stretch AKA Debian 9.0 will include a bunch of packages for people interested in digital forensics. The packages maintained within the Debian Forensics team which are new in the Debian/stretch release as compared to […]

Repeating what I did for the last Debian releases with the #newinwheezy and #newinjessie games it’s time for the #newinstretch game:

Debian/stretch AKA Debian 9.0 will include a bunch of packages for people interested in digital forensics. The packages maintained within the Debian Forensics team which are new in the Debian/stretch release as compared to Debian/jessie (and ignoring jessie-backports):

  • bruteforce-salted-openssl: try to find the passphrase for files encrypted with OpenSSL
  • cewl: custom word list generator
  • dfdatetime/python-dfdatetime: Digital Forensics date and time library
  • dfvfs/python-dfvfs: Digital Forensics Virtual File System
  • dfwinreg: Digital Forensics Windows Registry library
  • dislocker: read/write encrypted BitLocker volumes
  • forensics-all: Debian Forensics Environment – essential components (metapackage)
  • forensics-colorize: show differences between files using color graphics
  • forensics-extra: Forensics Environment – extra console components (metapackage)
  • hashdeep: recursively compute hashsums or piecewise hashings
  • hashrat: hashing tool supporting several hashes and recursivity
  • libesedb(-utils): Extensible Storage Engine DB access library
  • libevt(-utils): Windows Event Log (EVT) format access library
  • libevtx(-utils): Windows XML Event Log format access library
  • libfsntfs(-utils): NTFS access library
  • libfvde(-utils): FileVault Drive Encryption access library
  • libfwnt: Windows NT data type library
  • libfwsi: Windows Shell Item format access library
  • liblnk(-utils): Windows Shortcut File format access library
  • libmsiecf(-utils): Microsoft Internet Explorer Cache File access library
  • libolecf(-utils): OLE2 Compound File format access library
  • libqcow(-utils): QEMU Copy-On-Write image format access library
  • libregf(-utils): Windows NT Registry File (REGF) format access library
  • libscca(-utils): Windows Prefetch File access library
  • libsigscan(-utils): binary signature scanning library
  • libsmdev(-utils): storage media device access library
  • libsmraw(-utils): split RAW image format access library
  • libvhdi(-utils): Virtual Hard Disk image format access library
  • libvmdk(-utils): VMWare Virtual Disk format access library
  • libvshadow(-utils): Volume Shadow Snapshot format access library
  • libvslvm(-utils): Linux LVM volume system format access librar
  • plaso: super timeline all the things
  • pompem: Exploit and Vulnerability Finder
  • pytsk/python-tsk: Python Bindings for The Sleuth Kit
  • rekall(-core): memory analysis and incident response framework
  • unhide.rb: Forensic tool to find processes hidden by rootkits (was already present in wheezy but missing in jessie, available via jessie-backports though)
  • winregfs: Windows registry FUSE filesystem

Join the #newinstretch game and present packages and features which are new in Debian/stretch.




Debian stretch: changes in util-linux #newinstretch

2017-05-19T12:22:10Z

We’re coming closer to the Debian/stretch stable release and similar to what we had with #newinwheezy and #newinjessie it’s time for #newinstretch! Hideki Yamane already started the game by blogging about GitHub’s Icon font, fonts-octicons and Arturo Borrero Gonzalez wrote a nice article about nftables in Debian/stretch. One package that isn’t new but its tools […]We’re coming closer to the Debian/stretch stable release and similar to what we had with #newinwheezy and #newinjessie it’s time for #newinstretch! Hideki Yamane already started the game by blogging about GitHub’s Icon font, fonts-octicons and Arturo Borrero Gonzalez wrote a nice article about nftables in Debian/stretch. One package that isn’t new but its tools are used by many of us is util-linux, providing many essential system utilities. We have util-linux v2.25.2 in Debian/jessie and in Debian/stretch there will be util-linux >=v2.29.2. There are many new options available and we also have a few new tools available. Tools that have been taken over from other packages last: used to be shipped via sysvinit-utils in Debian/jessie lastb: used to be shipped via sysvinit-utils in Debian/jessie mesg: used to be shipped via sysvinit-utils in Debian/jessie mountpoint: used to be shipped via initscripts in Debian/jessie sulogin: used to be shipped via sysvinit-utils in Debian/jessie New tools lsipc: show information on IPC facilities, e.g.: root@ff2713f55b36:/# lsipc RESOURCE DESCRIPTION LIMIT USED USE% MSGMNI Number of message queues 32000 0 0.00% MSGMAX Max size of message (bytes) 8192 - - MSGMNB Default max size of queue (bytes) 16384 - - SHMMNI Shared memory segments 4096 0 0.00% SHMALL Shared memory pages 18446744073692774399 0 0.00% SHMMAX Max size of shared memory segment (bytes) 18446744073692774399 - - SHMMIN Min size of shared memory segment (bytes) 1 - - SEMMNI Number of semaphore identifiers 32000 0 0.00% SEMMNS Total number of semaphores 1024000000 0 0.00% SEMMSL Max semaphores per semaphore set. 32000 - - SEMOPM Max number of operations per semop(2) 500 - - SEMVMX Semaphore max value 32767 - - lslogins: display information about known users in the system, e.g.: root@ff2713f55b36:/# lslogins UID USER PROC PWD-LOCK PWD-DENY LAST-LOGIN GECOS 0 root 2 0 1 root 1 daemon 0 0 1 daemon 2 bin 0 0 1 bin 3 sys 0 0 1 sys 4 sync 0 0 1 sync 5 games 0 0 1 games 6 man 0 0 1 man 7 lp 0 0 1 lp 8 mail 0 0 1 mail 9 news 0 0 1 news 10 uucp 0 0 1 uucp 13 proxy 0 0 1 proxy 33 www-data 0 0 1 www-data 34 backup 0 0 1 backup 38 list 0 0 1 Mailing List Manager 39 irc 0 0 1 ircd 41 gnats 0 0 1 Gnats Bug-Reporting System (admin) 100 _apt 0 0 1 65534 nobody 0 0 1 nobody lsns: list system namespaces, e.g.: root@ff2713f55b36:/# lsns NS T[...]



Debugging a mystery: ssh causing strange exit codes?

2017-05-18T14:50:00Z

Recently we had a WTF moment at a customer of mine which is worth sharing. In an automated deployment procedure we’re installing Debian systems and setting up MySQL HA/Scalability. Installation of the first node works fine, but during installation of the second node something weird is going on. Even though the deployment procedure reported that […] Recently we had a WTF moment at a customer of mine which is worth sharing. In an automated deployment procedure we’re installing Debian systems and setting up MySQL HA/Scalability. Installation of the first node works fine, but during installation of the second node something weird is going on. Even though the deployment procedure reported that everything went fine: it wasn’t fine at all. After bisecting to the relevant command lines where it’s going wrong we identified that the failure is happening between two ssh/scp commands, which are invoked inside a chroot through a shell wrapper. The ssh command caused a wrong exit code showing up: instead of bailing out with an error (we’re running under ‘set -e‘) it returned with exit code 0 and the deployment procedure continued, even though there was a fatal error. Initially we triggered the bug when two ssh/scp command lines close to each other were executed, but I managed to find a minimal example for demonstration purposes: # cat ssh_wrapper chroot << "EOF" / /bin/bash ssh root@localhost hostname >/dev/null exit 1 EOF echo "return code = $?" What we’d expect is the following behavior, receive exit code 1 from the last command line in the chroot wrapper: # ./ssh_wrapper return code = 1 But what we actually get is exit code 0: # ./ssh_wrapper return code = 0 Uhm?! So what’s going wrong and what’s the fix? Let’s find out what’s causing the problem: # cat ssh_wrapper chroot << "EOF" / /bin/bash ssh root@localhost command_does_not_exist >/dev/null 2>&1 exit "$?" EOF echo "return code = $?" # ./ssh_wrapper return code = 127 Ok, so if we invoke it with a binary that does not exist we properly get exit code 127, as expected. What about switching /bin/bash to /bin/sh (which corresponds to dash here) to make sure it’s not a bash bug: # cat ssh_wrapper chroot << "EOF" / /bin/sh ssh root@localhost hostname >/dev/null exit 1 EOF echo "return code = $?" # ./ssh_wrapper return code = 1 Oh, but that works as expected!? When looking at this behavior I had the feeling that something is going wrong with file descriptors. So what about wrapping the ssh command line within different tools? No luck with `stdbuf -i0 -o0 -e0 ssh root@localhost hostname`, nor with `script -c “ssh root@localhost hostname” /dev/null` and also not with `socat EXEC:”ssh root@localhost hostname” STDIO`. But it works under unbuffer(1) from the expect package: # cat ssh_wrapper chroot << "EOF" / /bin/bash unbuffer ssh root@localhost hostname >/dev/null exit 1 EOF echo "return code = $?" # ./ssh_wrapper return code = 1 So my bet on something with the file descriptor handling was right. Going through the ssh manpage, what about using ssh’s `-n` option to prevent reading from standard input (stdin)? # cat ssh_wrapper chroot << "EOF" / /bin/bash ssh -n root@localhost hostname >/dev/null exit 1 EOF echo "return code = $?" # ./ssh_wrapper return code = 1 Bingo! Quoting ssh(1): -n Redirects stdin from /dev/null (actually, prevents reading from stdin). This must be used when ssh is run in the background. A common trick is to use this to run X11 programs on a remote machine. For example, ssh -n shadows.cs.hut.fi emacs & will start an emacs on shadows.cs.hut.fi, and the X11 connection will be[...]



Revisiting 2016

2017-01-04T13:58:54Z

Mainly to recall what happened last year and to give thoughts and planning for upcoming year(s) I’m once again revisiting the last year (previous years: 2015, 2014, 2013 + 2012). Here we go: Events: attended three conferences (OSDC in Germany, Linuxdays Graz + Debconf16 in Capetown) invited panel participant at Digitaldialog Privacy Technology / Open […]Mainly to recall what happened last year and to give thoughts and planning for upcoming year(s) I’m once again revisiting the last year (previous years: 2015, 2014, 2013 + 2012). Here we go: Events: attended three conferences (OSDC in Germany, Linuxdays Graz + Debconf16 in Capetown) invited panel participant at Digitaldialog Privacy Technology / Open Source: bunch of Gerrit, Jenkins, Docker, Debian + jenkins-debian-glue related work digged further into automated testing, especially with goss + sikuli (hope to have something available I can talk about in public in 2017/2018) sadly another year without an official stable Grml release (but we made progress in Q4 and a new stable release should be available in Q1/2017) Music: Bought a Cajón, my kids loving it as much as I do :) Played the drums more often in the beginning of 2016, went down close to zero in the second half of 2016 (meh) Sports: Bought a unicycle and started to learn to ride it in summer (got stuck since then without progress) Played less Badminton (as expected since I skipped the whole summer holidays season, but went kind of regularly during winter and summer terms otherwise) and played close to never Table Tennis (mainly due to date collisions with my sparring partner but also due to other time constraints), though managed to keep my handicap more or less in both sports Business: Third year of business with SynPro Solutions, very happy with what we achieved and very glad to have such fantastic partners Started to gather more metrics around my work related to Grml Solutions, SynPro Solutions + Grml-Forensic for better (capacity) planning (that’s something I’d like to talk about in public at some point) Personal: Reading activities was mainly around newspapers and articles, sadly not so many books First year of kindergarten for my older daughter, this brought us into a school-like schedule I wasn’t used to anymore (I’m finally starting to adopt to it though, also related to capacity planning efforts) Took two months of child care time (great time!), later on also taking care of my kids every Monday and half of Tuesday – this turned out to be way more stressful than expected (having just ~30 hours left per week for normal working hours and all my business duties, this is also the main reason why I started with the metrics thingy) Conclusion: 2016 was quite different to previous years, mainly because of being very time-constrained on all sides. One of the most challenging years overall. I’m planning to change quite some things for 2017 (started to do so already in Q4/2016), hoping for the best. [...]



Event: Digitaldialog Privacy

2016-11-25T09:40:54Z

Digitaldialog ist eine Veranstaltungsreihe der Steirischen Wirtschaftsförderung SFG. Am Dienstag den 29.11. findet der Digitaldialog zum Thema Privacy statt. Ich wurde eingeladen an der Podiumsdiskussion teilzunehmen und freue mich auf interessante Fragen aus dem Publikum. :) Weitere Informationen zum Event gibt es auf der SFG-Website und im Event-Flyer (PDF). Datum: Dienstag, 29.11.2016 ab 16:00 Uhr […]

Digitaldialog ist eine Veranstaltungsreihe der Steirischen Wirtschaftsförderung SFG. Am Dienstag den 29.11. findet der Digitaldialog zum Thema Privacy statt. Ich wurde eingeladen an der Podiumsdiskussion teilzunehmen und freue mich auf interessante Fragen aus dem Publikum. :)

Weitere Informationen zum Event gibt es auf der SFG-Website und im Event-Flyer (PDF).

  • Datum: Dienstag, 29.11.2016 ab 16:00 Uhr
  • Veranstalter: SFG, Infonova, Evolaris, IBC, Campus02, Kleine Zeitung, APA
  • Ort: Seering 10, 8141 Unterpremstätten, IBC Graz, Hotel Ramada
  • Eintritt frei