tag:blogger.com,1999:blog-55250443094103762282024-03-14T06:15:58.327+01:00unix-freakMartin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.comBlogger31125tag:blogger.com,1999:blog-5525044309410376228.post-32443668640423948162015-09-08T16:53:00.000+02:002015-09-09T12:03:29.691+02:00Puppet 4 - Examples - Data TypesPuppet 4 has some new functionality. Within the next few blog posts I will give some examples on how to use the new functionality.<br />
The first post covers the new Data Type system.<br />
<br />
<a name='more'></a>Data Types for parameterized classes<br />
<br />
Let's assume that you want to have a parameterized ssh class where users of your module might choose whether the server side should be installed.<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">class ssh (</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> $server = true, </span><br />
<span style="font-family: "Courier New", Courier, monospace;">){</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> ...</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> if $server {</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> ... </span><br />
<span style="font-family: "Courier New", Courier, monospace;"> }</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> .. </span><br />
<span style="font-family: "Courier New", Courier, monospace;">} </span><br />
<br />
And now we use the module:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">class { 'ssh':</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> server => 'false',</span><br />
<span style="font-family: "Courier New", Courier, monospace;">} </span><br />
<br />
In this example Puppet will verify the String 'false' which will result in a boolean value of 'true'.<br />
This is not what we expected.<br />
<br />
To avoid further misuse of the module we can add Data Types:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">class ssh (</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> <b>Boolean</b> $server = true, </span><br />
<span style="font-family: "Courier New", Courier, monospace;">){</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> ...</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> if $server {</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> ...</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> }</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> ... </span><br />
<span style="font-family: "Courier New", Courier, monospace;">}</span><br />
<br />
If we now use the same class declaration from above we will receive the following error message:<br />
<style type="text/css">p { margin-bottom: 0.1in; line-height: 120%; }</style> <br />
<div style="margin-bottom: 0.12in;">
<span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: small;"><span style="color: black;"><span style="letter-spacing: -0.1pt;">Error: Expected parameter 'server' of 'Class[Ssh]' to have type </span><span style="letter-spacing: -0.1pt;"><b>Boolean, got String</b></span><span style="letter-spacing: -0.1pt;"> at ssh.pp:2 on node puppetmaster.example.net</span></span></span></span></div>
<div style="margin-bottom: 0.12in;">
<span style="font-size: small;"><span style="font-family: inherit;"><span style="color: black;"><span style="letter-spacing: -0.1pt;"><br />
</span></span></span></span></div>
<br />
What Data Types are available?<br />
Puppet has Core Data Types and Abstract Data Types.<br />
The Core Data Types are the following:<br />
<ul>
<li>String</li>
<li>Integer</li>
<li>Float</li>
<li>Numeric</li>
<li>Boolean</li>
<li>Array</li>
<li>Hash</li>
<li>Regexp</li>
<li>Undef</li>
<li>Default</li>
</ul>
Integers are identified by a number (with or without minus sign) and no decimal point.<br />
<span style="font-family: "Courier New", Courier, monospace;">Integer $var1 = 1</span><br />
<span style="font-family: "Courier New", Courier, monospace;">Integer $var2 = -3 </span> <br />
Floats are identified by having a decimal point<br />
<span style="font-family: "Courier New", Courier, monospace;">Float $var3 = 1.0</span><br />
<span style="font-family: "Courier New", Courier, monospace;">Float $var4 = 0.2 </span><br />
Please note that you need to provide the trailing 0<br />
<br />
Most of the other Data Types explain themselves.<br />
The Default Data Type is something special:<br />
The Default Data Type can be used in case statements and selectors:<br />
<span style="font-family: "Courier New", Courier, monospace;">$real_server = $server ? {</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> Boolean => $server,</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> String => str2bool($server),</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> Default => true,</span><br />
<span style="font-family: "Courier New", Courier, monospace;">}</span><br />
Besides specifying the Data Types one can also specify Ranges of validity:<br />
<span style="font-family: "Courier New", Courier, monospace;">class ssh (</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> Integer[1,1024] $listen_port = 22, </span><br />
<span style="font-family: "Courier New", Courier, monospace;">){</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> ... </span><br />
<span style="font-family: "Courier New", Courier, monospace;">} </span><br />
<span style="font-family: inherit;">This will check that $listen_port is set to an Integer value within 1 and 1024 and will fail if the boundary limits are not met. </span><br />
The Abstract Data Types are the following:<br />
<ul>
<li>Scalar</li>
<li>Collection</li>
<li>Variant</li>
<li>Data</li>
<li>Pattern</li>
<li>Enum</li>
<li>Tuple</li>
<li>Struct</li>
<li>Optional</li>
<li>Catalogentry</li>
<li>Type</li>
<li>Any</li>
<li>Callable</li>
</ul>
One will mostly use Abstract Types which are built upon other Data Types.<br />
e.g. you want to express the Data Types of the content of an Array or an Hash:<br />
<span style="font-family: "Courier New", Courier, monospace;">class ntp (</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> Array[String] $ntp_servers = ['pool.ntp.org'], </span><br />
<span style="font-family: "Courier New", Courier, monospace;">){</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> ...</span><br />
<span style="font-family: "Courier New", Courier, monospace;">}</span><br />
<span style="font-family: inherit;">A more complex example which takes a hash map:</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$hash_map = {<br />
'ben' => {<br />
uid => 2203,<br />
home => '/home/ben',<br />
},<br />
'jones' => {<br />
uid => 2204,<br />
home => 'home/jones',<br />
}<br />
}</span><br />
<span style="font-family: "Courier New", Courier, monospace;">class usermanagement (</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> Hash[String, Struct[{ uid => Integer, home => Pattern[/^\/.*/]}]] $hash </span><br />
<span style="font-family: "Courier New", Courier, monospace;">) {</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> $keys = keys($hash)<br />
$keys.each |$single_key| {<br />
users::define{ $single_key:<br />
uid => $hash[$single_key]['uid'],<br />
home => $hash[$single_key]['home'],<br />
}<br />
} </span><br />
<span style="font-family: "Courier New", Courier, monospace;">}</span><br />
<span style="font-family: "Courier New", Courier, monospace;">define users::define (<br />
Integer $uid,<br />
Pattern[/^\/.*/] $home,<br />
){<br />
notify { "User: ${title}, UID: ${uid}, HOME: ${home}": }<br />
}</span><br />
<span style="font-family: "Courier New", Courier, monospace;">class { 'usermanagement':</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> hash => $hash_map,</span><br />
<span style="font-family: "Courier New", Courier, monospace;">} </span> <br />
<span style="font-family: inherit;">I<span style="font-family: inherit;">n case that one would like to allow certain words only, the Enum Data Type can be easily used:</span></span><br />
<span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: "Courier New", Courier, monospace;">class ssh (</span></span></span><br />
<span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: "Courier New", Courier, monospace;"> Enum['*','::1','127.0.0.1'] $listen_ip = '*', </span></span></span><br />
<span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: "Courier New", Courier, monospace;">){</span></span></span><br />
<span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: "Courier New", Courier, monospace;"> ...</span></span></span><br />
<span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: "Courier New", Courier, monospace;">} </span> </span></span><br />
<span style="font-family: inherit;"><span style="font-family: inherit;"> The Optional Data Type <span style="font-family: inherit;">describes parameters which <span style="font-family: inherit;">can be set to undef.</span></span></span></span><br />
<br />
<span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: inherit;">The next posting will cover Puppet 4 functions. </span></span></span></span>Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0Berlin, Germany52.520006599999988 13.40495399999997552.210736099999991 12.759506999999974 52.829277099999985 14.050400999999976tag:blogger.com,1999:blog-5525044309410376228.post-60256752496149597502015-05-01T20:08:00.000+02:002015-05-01T20:17:08.614+02:00Puppet Enterprise 3.8 Installation - steps and fixesThe new Puppet Enterprise Version 3.8 is out since a couple of days.<br />
It now supports Puppet 4 officially.<br />
On mailinglist people were telling about installation issues.<br />
This is a summary with fixes for SLES 11 and 12, CentOS 6 and 7, Ubuntu 12.04 and 14.10, Debian 6 and 7.<br />
Seems as if Debian 8 is still to new for PE 3.8.<br />
<br />
<br />
<a name='more'></a><br />
<div class="p1">
Hi,</div>
<div class="p2">
<br /></div>
<div class="p1">
this is a summary of fresh PE 3.8.0 installation (no update) on SLES 11 and 12, Ubuntu 14 and 12, Debian 7 and CentOS 6 and 7</div>
<div class="p2">
<br /></div>
<div class="p1">
All machines were installed using standard installation disks without adding additional software or enabling other repositories.</div>
<div class="p1">
All machines were deployed under VirtualBox on a MacBook running OS 10.9</div>
<div class="p2">
<br /></div>
<div class="p1">
<b>SLES 12:</b></div>
<div class="p2">
<br /></div>
<div class="p1">
no problems.</div>
<div class="p1">
Just make sure to set lang and LC_CTYPE to en_US.utf8</div>
<blockquote class="tr_bq">
export LANG=en_US.utf8<br />
export LANG=en_US.utf8</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
<b>SLES 11:</b></div>
<div class="p2">
<br /></div>
<div class="p1">
Make sure to set LANG and LC_CTYPE to en_US.utf8</div>
<blockquote class="tr_bq">
export LANG=en_US.utf8<br />
export LC_CTYPE=en_US.utf8</blockquote>
<div class="p1">
When you see the following error message:</div>
<blockquote class="tr_bq">
Problem: nothing provides db43 needed by pe-libaprutil-1.5.3-1.pe.sles11.x86_64</blockquote>
<div class="p1">
One has to enable SLES SDK repo or install the db43 package manually:</div>
<div class="p1">
<br /></div>
<div class="p1">
<a href="https://www.novell.com/support/kb/doc.php?id=7015337">https://www.novell.com/support/kb/doc.php?id=7015337</a></div>
<div class="p1">
<a href="http://www.filewatcher.com/m/db43-4.3.29-125.17.x86_64.rpm.790869-0.html">http://www.filewatcher.com/m/db43-4.3.29-125.17.x86_64.rpm.790869-0.html</a></div>
<div class="p1">
<a href="http://suse.curingapneus.com.br/SLES-11-SP3-SDK-DVD-x86_64-GM-DVD1/suse/x86_64/">http://suse.curingapneus.com.br/SLES-11-SP3-SDK-DVD-x86_64-GM-DVD1/suse/x86_64/</a></div>
<div class="p1">
<br /></div>
<blockquote class="tr_bq">
rpm -ihv sle*.rpm<br />
or<br />
rpm -ihv db43*.rpm</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
Then the installer fails with error:</div>
<blockquote class="tr_bq">
** chkconfig pe-memcached on<br />
** service pe-httpd start<br />
Starting pe-httpd: pe-httpd.worker: apr_sockaddr_info_get() failed for linux-gd74<br />
pe-httpd.worker: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName<br />
(98)Address already in use: make_sock: could not bind to address 127.0.0.1:4434<br />
no listening sockets available, shutting down<br />
Unable to open logs<br />
..failed<br />
/tmp/pe-installer-KzjtkqLH/install/puppet-enterprise-installer: line 243: [: 18446744073709551615: integer expression expected</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
242 # Set kernel.shmmax if it's less than half the available memory size</div>
<div class="p1">
243 if [ "${t_postgres_shmmax_req?}" -gt ${t_postgres_shmmax_avail?} ]; then</div>
<div class="p1">
244 export t_manage_kernel_shmmax='y'</div>
<div class="p1">
245 fi</div>
<div class="p2">
<br /></div>
<div class="p1">
This is due to a bug in the installer. The following patch fixes this:</div>
<div class="p2">
<br /></div>
<blockquote class="tr_bq">
--- puppet-enterprise-installer.orig<span class="Apple-tab-span"> </span>2015-05-01 12:44:00.000000000 +0200<br />
+++ puppet-enterprise-installer<span class="Apple-tab-span"> </span>2015-05-01 12:28:00.000000000 +0200<br />
@@ -240,7 +240,7 @@<br />
t_postgres_shmmax_avail="$(/sbin/sysctl kernel.shmmax | awk '{ printf $3 }')"<br />
# Set kernel.shmmax if it's less than half the available memory size<br />
- if [ "${t_postgres_shmmax_req?}" -gt ${t_postgres_shmmax_avail?} ]; then<br />
+ if [ ${t_postgres_shmmax_req?} -gt ${t_postgres_shmmax_avail?} ]; then<br />
export t_manage_kernel_shmmax='y'<br />
fi<br />
fi</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
Afterwards the installer runs fine.</div>
<div class="p2">
<br /></div>
<div class="p2">
<br /></div>
<div class="p1">
<b>Ubuntu 14.10:</b></div>
<div class="p2">
<br /></div>
<div class="p1">
Make sure to set LANG and LC_CTYPE to en_US.UTF-8</div>
<blockquote class="tr_bq">
export LANG=en_US.UTF-8<br />
export LC_CTYPE=en_US.UTF-8</blockquote>
<div class="p1">
<br /></div>
<div class="p1">
Install packages which are not in PE installer dependency:</div>
<blockquote class="tr_bq">
apt-get install zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties</blockquote>
<div class="p1">
(Thanks to Kamil Keski for pointing to the missing packages)</div>
<div class="p2">
<br /></div>
<div class="p1">
If you do not install the above mentioned packages the installation fails:</div>
<blockquote class="tr_bq">
** HTTP_PROXY= http_proxy= HTTPS_PROXY= https_proxy= curl --tlsv1 -s https://puppet:8140<br />
** HTTP_PROXY= http_proxy= HTTPS_PROXY= https_proxy= curl --tlsv1 -s https://puppet:8140<br />
========================================================================<br />
** cp /opt/puppet/share/installer/install_log.lastrun.puppet.log /var/log/pe-installer<br />
** sed "s/^\(q_.*password\)=.*/#\1=REDACTED/g" < "/tmp/pe-installer-H8P227I5/install/answers.lastrun.puppet" > "/var/log/pe-installer/answers.install"<br />
!! ERROR: The puppet master service failed to start within 120<br />
seconds; unable to proceed</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
from puppetserver.log:</div>
<blockquote class="tr_bq">
2015-05-01 10:37:09,665 ERROR [p.t.internal] shutdown-on-error triggered because of exception!<br />
java.lang.IllegalStateException: There was a problem adding a JRubyPuppet instance to the pool.<br />
...<br />
Caused by: org.jruby.embed.EvalFailedException: (Error) Cannot determine basic system flavour<br />
…<br />
2015-05-01 10:37:09,674 ERROR [p.t.internal] Error during service init!!!<br />
java.lang.IllegalStateException: Unable to borrow JRuby instance from pool<br />
…<br />
2015-05-01 10:37:09,680 INFO [p.t.internal] Beginning shutdown sequence<br />
2015-05-01 10:37:09,694 INFO [p.t.s.w.jetty9-service] Shutting down web server(s).<br />
2015-05-01 10:37:09,701 INFO [p.t.internal] Finished shutdown sequence</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
after installing packages Kamil mentioned in his email the pe-puppetserver process starts fine and the installer finished successfully.</div>
<div class="p2">
<br /></div>
<div class="p1">
<b>Ubuntu 12.10:</b></div>
<div class="p2">
<br /></div>
<div class="p1">
not supported</div>
<div class="p2">
<br /></div>
<div class="p1">
<b>Ubuntu 12.04:</b></div>
<div class="p2">
<br /></div>
<div class="p1">
Make sure to set LANG and LC_CTYPE to en_US.UTF-8</div>
<blockquote class="tr_bq">
export LANG=en_US.UTF-8<br />
export LC_CTYPE=en_US.UTF-8</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
apt-get update</div>
<div class="p1">
When you get Hash Sum mismatch error:</div>
<blockquote class="tr_bq">
rm -fr /var/lib/apt/lists/*<br />
apt-get update</blockquote>
<div class="p1">
http://askubuntu.com/questions/41605/trouble-downloading-packages-list-due-to-a-hash-sum-mismatch-error</div>
<div class="p2">
<br /></div>
<div class="p1">
Same issue as with Ubuntu 14.10:</div>
<blockquote class="tr_bq">
apt-get install zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
then PE installer works without errors.</div>
<div class="p2">
<br /></div>
<div class="p2">
<br /></div>
<div class="p1">
<b>Debian 6:</b></div>
<div class="p2">
<br /></div>
<div class="p1">
Make sure LANG and LC_CTYPE are set to en_US.UTF-8</div>
<blockquote class="tr_bq">
export LANG=en_US.UTF-8<br />
export LC_CTYPE=en_US.UTF-8</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
no problems with PE installer</div>
<div class="p2">
<br /></div>
<div class="p1">
<b>Debian 7:</b></div>
<div class="p2">
<br /></div>
<div class="p1">
Make sure LANG and LC_CTYPE are set to en_US.UTF-8</div>
<blockquote class="tr_bq">
export LANG=en_US.UTF-8<br />
export LC_CTYPE=en_US.UTF-8</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
no problems with PE installer</div>
<div class="p2">
<br /></div>
<div class="p1">
<b>CentOS 6:</b></div>
<div class="p2">
<br /></div>
<div class="p1">
Make sure LANG and LC_CTYPE are set to en_US.UTF-8</div>
<blockquote class="tr_bq">
export LANG=en_US.UTF-8<br />
export LC_CTYPE=en_US.UTF-8</blockquote>
<div class="p2">
<br /></div>
<div class="p1">
no problems with PE installer</div>
<div class="p2">
<br /></div>
<div class="p2">
<br /></div>
<div class="p1">
<b>CentOS 7:</b></div>
<div class="p2">
<br /></div>
<div class="p1">
Make sure LANG and LC_CTYPE are set to en_US.UTF-8</div>
<blockquote class="tr_bq">
export LANG=en_US.UTF-8<br />
export LC_CTYPE=en_US.UTF-8</blockquote>
<div class="p1">
Disable Firewall or allow port 3000 (installation) and Puppet ports</div>
<div class="p2">
<br /></div>
<br />
<div class="p1">
no problems with PE installer</div>
Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-16423683748053074162015-04-26T13:12:00.000+02:002015-04-26T13:12:53.420+02:00Puppet Enterprise on SLESSLES and OpenSuSE are supported operatingsystems by Puppet.<br />
But I always saw trouble when installing either the Puppet Master or the Agent on SLES.<br />
<br />
One needs to take care on locales, paranoid umask, required packages.<br />
<br />
<a name='more'></a><br />
1. locale<br />
<br />
Standard SLES installation will set locale to POSIX.<br />
This will break UTF-8 compatibility with some Puppet Modules.<br />
<br />
Instead: set locale to en_US.utf8 in your .bashrc file.<br />
<br />
2. paranoid umask<br />
<br />
The paranoid umask (0002) will cause severe trouble on SLES.<br />
Please lower the security constrints for umask.<br />
<br />
3. required packages<br />
<br />
When running SLES 11 one will see an error message regarding a missing package during installation:<br />
<blockquote class="tr_bq">
<span style="-webkit-text-stroke-width: 0px; background-color: white; color: rgba(0, 0, 0, 0.8); display: inline !important; float: none; font-family: 'Roboto Slab', 'Times New Roman', serif; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 19px; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: pre-wrap; widows: 1; word-spacing: 0px;">nothing provides db43, libgcc</span></blockquote>
You need to add the SDK channel to get the required package.<br />
<br />
What else is missing:<br />
<br />
- The Live Management package checkupdates does not know about zypper.<br />
- Modules from other authors might not have proper SLES support.<br />
<br />
I will add more articles with more findings. <br />
<br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0Berlin, Germany52.520006599999988 13.40495399999997552.210736099999991 12.759506999999974 52.829277099999985 14.050400999999976tag:blogger.com,1999:blog-5525044309410376228.post-67877409749049589662014-09-23T23:54:00.000+02:002014-09-23T23:54:31.939+02:00News from PuppetConf - Puppet 4 server and agentPuppet is working on new stuff for Puppet 4.x<br />
During PuppetConf they provided information of upcoming changes:<br />
<br />
<a name='more'></a><br />
<br />
<h2>
What will the Puppet server side be like?</h2>
<br />
- continue usage of feature flags (like future parser)<br />
- deprecation warnings are now a sign of upcoming maturity<br />
<br />
Upcoming changes cover Puppet Language, Environments and Components<br />
<br />
<b>Changes to Language</b>:<br />
<br />
- data validation built into parameterized classes<br />
<br />
<blockquote class="tr_bq">
<pre class="prettyprint"><code class="language-rb">define sinatra(String $regrets, Integer $amount) {
notice "$regrets, I had $amount, I did it my way. Do bi do bi doo..."
}
sinatra{ frank:
$regrets => regrets,
$amount => 2 # e.g. 'a few'
}</code></pre>
</blockquote>
http://puppet-on-the-edge.blogspot.com/2014/06/optionally-typed-parameters.html<br />
<br />
The sinatra define will validate that $regrets is a string and amount is an integer<br />
<br />
<br />
<b>Changes to Environments</b>:<br />
<br />
- we will have directory based environments.<br />
- we will see environments to be similar to modules<br />
<br />
<b>Changes to Components</b><br />
<br />
Puppet will break into more components (micro services).<br />
This allows you better separation and scaling.<br />
Of course one can run all components bundled on one single system.<br />
<br />
The puppet server side will be running on JVM with jruby.<br />
We will no longer cope with Apache/Passenger tweaking.<br />
This also leads to better Environment separation. And allow per environment pluginsync.<br />
<br />
<h2>
What will the Puppet Agent side be like?</h2>
<br />
<b>Facter</b><br />
<br />
Facter will be a native C++ application (cfacter).<br />
Old custom facts will still be possible by a Ruby-C++-API.<br />
Support for other languages (Perl, Python) is possible, but not yet implemented.<br />
<br />
<b>Puppet Agent</b><br />
<br />
Comes next. Plans are to got for C++, too.<br />
Puppet developers are eager to learn from community regarding masterless puppet.<br />
<br />
<h2>
Summary</h2>
Puppet Master is moving to JVM (https://github.com/puppetlabs/puppet-server)<br />
Facter is moving to C++.<br />
Agent is most likely to become also C++.<br />
<br />
Most stuff is now moving away fro Ruby.<br />
Puppet promized that Ruby is still the core and will be around for "a very long time" (Andy Parker @PuppetConf 2014).<br />
<br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0San Francisco, CA, USA37.7749295 -122.4194155000000137.373502 -123.06486250000002 38.176356999999996 -121.77396850000001tag:blogger.com,1999:blog-5525044309410376228.post-63518080382234838202014-09-04T10:23:00.000+02:002014-09-04T10:23:24.514+02:00Summary of Puppet 4 features and deprecationsPuppetLabs maintains a github repo where they manage documents regarding Puppet 4.x changes, deprecations and new features.<br />
This article is a summary on planned and implemented changes.<br />
<br />
<br />
<a name='more'></a><br />
https://github.com/puppetlabs/puppet-specifications<br />
<br />
<h4>
Environment</h4>
<br />
New environment configuration pattern: environmentpath<br />
<br />
Each environment directory<br />
- can have environment.conf - overwrite defaults for this special env<br />
- has manifests and modules<br />
<br />
Within manifests directory all files are processed in alphabetical order<br />
<br />
<br />
<h4>
Catalog compile</h4>
<br />
1. initial evaluation<br />
- load global variables<br />
- set node definitions<br />
2. selection of node definition<br />
3. evaluation of node definition<br />
4. catalog completion<br />
<br />
<h4>
Autoloading:</h4>
<br />
In Puppet3 everything is visible to everything.<br />
You can build references among different modules.<br />
<br />
In Puppet4 visibility is limited:<br />
<br />
- Puppet Runtime defines are visible to everything<br />
- everything in an environment is visible to everything in the same environment<br />
- modules without dependencies see all other modules<br />
- modules with dependency see the module it depends on<br />
<br />
<br />
<h4>
Types, Values, Variables</h4>
<br />
Puppet Types<br />
- Data Types<br />
- Integer<br />
- Float<br />
- Boolean<br />
- Regexp<br />
- String<br />
- Array<br />
- Hash<br />
- Catalog Types<br />
- Resource<br />
- Class<br />
- Abstract Types<br />
- Collection<br />
- Scalar<br />
- Numeric<br />
- CatalogEntry<br />
- Data<br />
- Tuple<br />
- Struct<br />
- Optional<br />
- Variant<br />
- Enum<br />
- Pattern<br />
- Any<br />
<br />
Platform Types<br />
- Callable<br />
- Type<br />
- Runtime<br />
- Undef<br />
- Default<br />
<br />
<h4>
Variables</h4>
<br />
strict naming:<br />
- start with lower case letter<br />
- hyphen and period are no longer supported<br />
- No upper case letter in initial position<br />
- no underscore in initial position<br />
<br />
Variable References<br />
optional (configurable) strict variable lookup in Puppet 4 - will become mandatory in Puppet 5<br />
<br />
Boolean conversion<br />
‘’ -> true (false in Puppet 3 !!!)<br />
undef -> false<br />
false -> false<br />
any other -> true<br />
<br />
<br />
<h4>
Scoping</h4>
<br />
- Top scope<br />
- Class scope<br />
- define scope<br />
- lambda scope<br />
- node scope<br />
<br />
Access control (not yet implemented)<br />
<br />
apply the keyword “private” to<br />
- class<br />
- define<br />
- parameter<br />
- variable<br />
<br />
Private classes may only be<br />
- included<br />
- required<br />
- instantiated (declared)<br />
- inherited<br />
- form relationship with private class<br />
from the module where it is declared<br />
References are possible<br />
<br />
Private defines may only be<br />
- instantiated (declared)<br />
- form relationship<br />
form the module where it is declared<br />
<br />
Private parameters may only be<br />
- referencing using variable syntax<br />
- referencing parameter via catalog lookup and accessing its parameter<br />
<br />
Private variable may only be<br />
- available in the scope where declared<br />
This is only useful for variables in top, node or class scope.<br />
It is not visible to other scopes.<br />
<br />
<br />
<h4>
Lexical Stuff</h4>
Unicode<br />
Puppet reads ASCII files.<br />
One may use Unicode escape mechanism (/uXXXX).<br />
In the Future it may be possible to specify encoding for a file.<br />
<br />
Reserved keywords:<br />
- false<br />
- true<br />
- undef<br />
- and<br />
- case<br />
- class<br />
- default<br />
- define<br />
- else<br />
- elsif<br />
- if<br />
- in<br />
- inherits<br />
- node<br />
- or<br />
- unless<br />
Reserved for future:<br />
- type<br />
- function<br />
- private<br />
- attr<br />
Reserved for types:<br />
- any, Any<br />
- hash, Hash<br />
- array, Array<br />
- integer, Integer<br />
- float, Float<br />
- collection, Collection<br />
- scalar, Scalar<br />
- resource, Resource<br />
- string, String<br />
- pattern, Pattern<br />
- boolean, Boolean<br />
- class, Class<br />
- type, Type<br />
- runtime, Runtime<br />
- numeric, Numeric<br />
- data, Data<br />
- catalogentry, catalogEntry, CatalogEntry<br />
- enum, Enum<br />
- variant, Variant<br />
- data, Data<br />
-struct, Struct<br />
- tuple, Tuple<br />
- optional, Optional<br />
- undef, Undef<br />
- default, Default<br />
<br />
<br />
<h4>
Definitions</h4>
<br />
Node definition changes. Node inheritance will now raise an error.<br />
<br />
Class definition remains as is<br />
<br />
Resource definition remains as is<br />
<br />
Resource defaults remain ss is<br />
<br />
<br />
<h4>
Deprecations</h4>
<br />
“class” is not a valid class name<br />
Upper case variable names<br />
Variables with underscore<br />
- Local var may have underscore<br />
- qualified var may not have an underscore in first position<br />
Access to non existing variables<br />
- turn on —strict-mode<br />
Access to class using upper case name/title<br />
+= and -= have den deprecated<br />
hyphens in names<br />
ruby dsl<br />
<br />
<br />
<br />
<h4>
Functions in Puppet</h4>
<br />
Autoloaded from Modules<br />
<modulepath>/<modulename>/functions/<func name>.pp<br />
e.g. function min in module test<br />
min.pp<br />
function test::min ($a, $b) { … }<br />
<br />
Unfold (match all elements of an array)<br />
$a = [1,2,3]<br />
foo(*$a) # same as foo(1, 2, 3)<br />
<br />
case $something {<br />
*$a: { # matches 1 or 2 or 3<br />
}<br />
}<br />
<br />
$b = [10, *$a, 20] # creates [10, 1, 2, 3, 20]<br />
<br />
<br />
<br />
<h4>
Puppet Function API</h4>
new API in Puppet 4.x<br />
Reason:<br />
in Puppet 3.x:<br />
- functions can not be private<br />
- are defined in Puppet::Parser::Functions scope<br />
- no automatic type checking<br />
<br />
In Puppet 3.x API:<br />
Puppet::Parser::Functions.newfunction<br />
- name<br />
- type<br />
- arity<br />
- doc<br />
<br />
In Puppet 4.x API:<br />
Puppet::Function.create_function<br />
<br />
Eample:<br />
fix numer of arguments:<br />
Puppet::Functions.create_function(:max) do<br />
def max(x,y)<br />
x >= y ? x : y<br />
end<br />
end<br />
<br />
flexible number of arguments:<br />
def myfunc (a, b, c, *d)<br />
<br />
Argument validation via dispatch (choose the proper method to use)<br />
dispatch :min do<br />
param 'Numeric', :a<br />
param 'Numeric', :b<br />
end<br />
<br />
dispatch :min_s do<br />
param 'String', :s1<br />
param 'String', :s2<br />
end<br />
def min(x,y)<br />
x <= y ? x : y<br />
end<br />
<br />
def min_s(x,y)<br />
cmp = (x.downcase <=> y.downcase)<br />
cmp <= 0 ? x : y<br />
end<br />
<br />
<!--more--><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-43819646249944513382014-08-07T13:28:00.001+02:002014-08-07T13:37:09.549+02:00OpenNebula Puppet types and providersWhen setting up OpenNebula one has to configure many steps.<br />
Since I do not like manual work and due to the reason that we already had the OpenNebula Puppet module available as OpenSource on GitHub, I started working on OpenNebula types and Providers.<br />
<br />
You now can declare your OpenNebula infrastructure in Puppet DSL:<br />
<br />
<a name='more'></a>All code can be found on the github project: https://github.com/epost-dev/opennebula-puppet-module<br />
<br />
This is a short summary on how to use the OpenNebula Puppet Module and the newly added Types and Providers.<br />
Please note: there is still some functionality missing and the providers do far too much one cli command calls.<br />
All work is based on Ken Barbers initial OpenNebula Puppet Module.<br />
<br />
<div class="code-body highlight">
<blockquote class="tr_bq">
<div class="line js-file-line" id="LC1">
<span class="c"># example usage for opennebula puppet module</span>
</div>
<div class="line js-file-line" id="LC2">
<span class="c">#</span>
</div>
<div class="line js-file-line" id="LC3">
<span class="c"># 1. Running as OpenNebula Master with Apache and mod_passenger and Sunstone</span>
</div>
<div class="line js-file-line" id="LC4">
<span class="c"># using kvm and 802.1q VLAN networking:</span><br />
# </div>
<div class="line js-file-line" id="LC5">
<span class="k">class</span> <span class="p">{</span> <span class="na">one</span><span class="p">:</span>
</div>
<div class="line js-file-line" id="LC6">
<span class="na">oned</span> <span class="o">=></span> <span class="k">true</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC7">
<span class="na">sunstone_passenger</span> <span class="o">=></span> <span class="k">true</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC8">
<span class="p">}</span>
</div>
<div class="line js-file-line" id="LC10">
<span class="c"># 2. running opennebula vm wirt side</span></div>
#
<br />
<div class="line js-file-line" id="LC11">
<span class="k">class</span> <span class="p">{</span> <span class="na">one</span><span class="p">:</span> <span class="p">}</span>
</div>
<div class="line js-file-line" id="LC13">
<span class="c"># Usage of opennebula puppet resource types</span>
</div>
<div class="line js-file-line" id="LC13">
<span class="c">#</span>
</div>
<div class="line js-file-line" id="LC13">
<span class="c"># Declare OpenNebula Virtual Network:</span>
</div>
<div class="line js-file-line" id="LC13">
<span class="c"># </span>
</div>
<div class="line js-file-line" id="LC14">
<span class="na">onevnet</span> <span class="p">{</span> <span class="s">'<name>'</span><span class="p">:</span>
</div>
<div class="line js-file-line" id="LC15">
<span class="na">ensure</span> <span class="o">=></span> <span class="k">present</span> <span class="o">|</span> <span class="k">absent</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC16">
<span class="c"> # name of the bridge to use</span>
</div>
<div class="line js-file-line" id="LC17">
<span class="na">bridge</span> <span class="o">=></span> <span class="s">'basebr0'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC18">
<span class="c"> # name of the physical interface on which the bridge wiull run</span>
</div>
<div class="line js-file-line" id="LC19">
<span class="na">phydev</span> <span class="o">=></span> <span class="s">'br0'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC20">
<span class="na">type</span> <span class="o">=></span> <span class="s">'ranged'</span> <span class="o">|</span> <span class="s">'fixed'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC21">
<span class="c"> # for ranged networking:</span>
</div>
<div class="line js-file-line" id="LC22">
<span class="na">network_address</span> <span class="o">=></span> <span class="s">'10.0.2.0'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC23">
<span class="na">network_mask</span> <span class="o">=></span> <span class="s">'255.255.0.0'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC24">
<span class="c"> # if you want to restrict network usage by opennebula:</span>
</div>
<div class="line js-file-line" id="LC25">
<span class="na">network_start</span> <span class="o">=></span> <span class="s">'10.0.2.100'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC26">
<span class="na">network_end</span> <span class="o">=></span> <span class="s">'10.0.2.240'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC27">
<span class="na">dns_servers</span> <span class="o">=></span> <span class="p">[</span><span class="s">'8.8.8.8'</span><span class="p">,</span> <span class="s">'4.4.4.4'</span><span class="p">],</span>
</div>
<div class="line js-file-line" id="LC28">
<span class="na">gateway</span> <span class="o">=></span> <span class="s">'10.0.2.1'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC29">
<span class="c"> # select your network type</span>
</div>
<div class="line js-file-line" id="LC30">
<span class="na">model</span> <span class="o">=></span> <span class="s">'vlan'</span> <span class="o">|</span> <span class="s">'ebtables'</span> <span class="o">|</span> <span class="s">'ovswitch'</span> <span class="o">|</span> <span class="s">'vmware'</span> <span class="o">|</span> <span class="s">'dummy'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC31">
<span class="c"> # add vlanid </span>
</div>
<div class="line js-file-line" id="LC32">
<span class="na">vlanid</span> <span class="o">=></span> <span class="s">'1550'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC33">
<span class="p">}</span>
</div>
<div class="line js-file-line" id="LC34">
# Declare OpenNebula Datastores
</div>
<div class="line js-file-line" id="LC34">
#
</div>
<div class="line js-file-line" id="LC35">
<span class="na">onedatastore</span> <span class="p">{</span> <span class="s">'<name>'</span><span class="p">:</span>
</div>
<div class="line js-file-line" id="LC36">
<span class="na">ensure</span> <span class="o">=></span> <span class="k">present</span> <span class="o">|</span> <span class="k">absent</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC37">
<span class="na">type</span> <span class="o">=></span> <span class="s">'images'</span> <span class="o">|</span> <span class="s">'system'</span> <span class="o">|</span><span class="err"> </span><span class="s">'files'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC38">
<span class="na">dm</span> <span class="o">=></span> <span class="s">'filesystem'</span> <span class="o">|</span> <span class="s">'vmware'</span> <span class="o">|</span><span class="err"> </span><span class="s">'iscsi'</span> <span class="o">|</span> <span class="s">'lvm'</span> <span class="o">|</span> <span class="s">'vmfs'</span> <span class="o">|</span><span class="err"> </span><span class="s">'ceph'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC39">
<span class="na">tm</span> <span class="o">=></span> <span class="s">'shared'</span> <span class="o">|</span><span class="err"> </span><span class="s">'ssh'</span> <span class="o">|</span> <span class="s">'qcow2'</span> <span class="o">|</span> <span class="s">'iscsi'</span> <span class="o">|</span> <span class="s">'lvm'</span> <span class="o">|</span> <span class="s">'vmfs'</span> <span class="o">|</span> <span class="s">'ceph'</span> <span class="o">|</span><span class="err"> </span><span class="s">'dummy'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC40">
<span class="na">disktype</span> <span class="o">=></span> <span class="s">'file'</span> <span class="o">|</span><span class="err"> </span><span class="s">'block'</span> <span class="o">|</span><span class="err"> </span><span class="s">'rdb'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC41">
<span class="p">}</span>
</div>
<div class="line js-file-line" id="LC42">
# Declare OpenNebula Hosts
</div>
<div class="line js-file-line" id="LC42">
#
</div>
<div class="line js-file-line" id="LC43">
<span class="na">onehost</span> <span class="p">{</span> <span class="s">'<name>'</span><span class="p">:</span>
</div>
<div class="line js-file-line" id="LC44">
<span class="na">ensure</span> <span class="o">=></span> <span class="k">present</span> <span class="o">|</span> <span class="k">absent</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC45">
<span class="na">im_mad</span> <span class="o">=></span> <span class="s">'kvm'</span> <span class="o">|</span> <span class="s">'xen'</span> <span class="o">|</span> <span class="s">'vmware'</span> <span class="o">|</span> <span class="s">'ec2'</span> <span class="o">|</span> <span class="s">'ganglia'</span> <span class="o">|</span> <span class="s">'dummy'</span> <span class="o">|</span> <span class="s">'custom'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC46">
<span class="na">vm_mad</span> <span class="o">=></span> <span class="s">'kvm'</span> <span class="o">|</span> <span class="s">'xen'</span> <span class="o">|</span> <span class="s">'vmware'</span> <span class="o">|</span> <span class="s">'ec2'</span> <span class="o">|</span> <span class="s">'dummy'</span> <span class="o">|</span> <span class="s">'custom'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC47">
<span class="na">vn_mad</span> <span class="o">=></span> <span class="s">'dummy'</span> <span class="o">|</span> <span class="s">'firewall'</span> <span class="o">|</span> <span class="s">'vlan'</span> <span class="o">|</span> <span class="s">'ebtables'</span> <span class="o">|</span> <span class="s">'ovswitch'</span> <span class="o">|</span> <span class="s">'vmware'</span> <span class="o">|</span><span class="err"> </span><span class="s">'custom'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC48">
<span class="p">}</span>
</div>
<div class="line js-file-line" id="LC49">
# Declare OpenNebula clusters
</div>
<div class="line js-file-line" id="LC49">
#
</div>
<div class="line js-file-line" id="LC50">
<span class="na">onecluster</span> <span class="p">{</span> <span class="s">'<name>'</span><span class="p">:</span>
</div>
<div class="line js-file-line" id="LC51">
<span class="na">ensure</span> <span class="o">=></span> <span class="k">present</span> <span class="o">|</span> <span class="k">absent</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC52">
<span class="na">hosts</span> <span class="o">=></span> <span class="p">[</span> <span class="s">'host1'</span><span class="p">,</span> <span class="s">'host2'</span><span class="p">,</span><span class="err">...</span><span class="p">],</span>
</div>
<div class="line js-file-line" id="LC53">
<span class="na">vnets</span> <span class="o">=></span> <span class="p">[</span> <span class="s">'vnet1'</span><span class="p">,</span> <span class="s">'vnet2'</span><span class="p">,</span> <span class="err">...</span><span class="p">],</span>
</div>
<div class="line js-file-line" id="LC54">
<span class="na">datastores</span> <span class="o">=></span> <span class="p">[</span> <span class="s">'ds1'</span><span class="p">,</span> <span class="s">'ds2'</span><span class="p">,</span> <span class="err">...</span><span class="p">],</span>
</div>
<div class="line js-file-line" id="LC55">
<span class="p">}</span>
</div>
<div class="line js-file-line" id="LC56">
# Declare OpenNebula images
</div>
<div class="line js-file-line" id="LC56">
#
</div>
<div class="line js-file-line" id="LC57">
<span class="na">oneimage</span> <span class="p">{</span><span class="err"> </span><span class="s">'<name>'</span><span class="p">:</span>
</div>
<div class="line js-file-line" id="LC58">
<span class="na">ensure</span> <span class="o">=></span> <span class="k">present</span> <span class="o">|</span> <span class="k">absent</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC59">
<span class="na">datastore</span> <span class="o">=></span> <span class="s">'default'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC60">
<span class="na">description</span> <span class="o">=></span> <span class="s">'Image description'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC61">
<span class="na">type</span> <span class="o">=></span> <span class="s">'os'</span> <span class="o">|</span><span class="err"> </span><span class="s">'cdrom'</span> <span class="o">|</span> <span class="s">'datablock'</span> <span class="o">|</span> <span class="s">'kernel'</span> <span class="o">|</span> <span class="s">'ramdisk'</span> <span class="o">|</span> <span class="s">'context'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC62">
<span class="na">persistent</span> <span class="o">=></span> <span class="s">'yes'</span> <span class="o">|</span><span class="err"> </span><span class="s">'no'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC63">
<span class="na">dev_prefix</span> <span class="o">=></span> <span class="s">'hd'</span> <span class="o">|</span> <span class="s">'sd'</span> <span class="o">|</span> <span class="s">'xvd'</span> <span class="o">|</span> <span class="s">'vd'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC64">
<span class="na">target</span> <span class="o">=></span> <span class="s">'hda'</span> <span class="o">|</span> <span class="s">'hdb'</span> <span class="o">|</span> <span class="s">'sda'</span> <span class="o">|</span> <span class="s">'sdb'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC65">
<span class="na">path</span> <span class="o">=></span> <span class="s">'/tmp/image_file'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC66">
<span class="na">driver</span> <span class="o">=></span> <span class="s">'raw'</span> <span class="o">|</span><span class="err"> </span><span class="s">'qcow2'</span> <span class="o">|</span> <span class="s">'tap:aio'</span> <span class="o">|</span> <span class="s">'file:'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC67">
<span class="c"> # non file based images</span>
</div>
<div class="line js-file-line" id="LC68">
<span class="na">source</span> <span class="o">=></span> <span class="s">''</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC69">
<span class="na">size</span> <span class="o">=></span> <span class="s">'11200'</span><span class="c"> # 11.2 GB</span>
</div>
<div class="line js-file-line" id="LC70">
<span class="na">fstype</span> <span class="o">=></span> <span class="s">'ext3'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC71">
<span class="p">}</span>
</div>
<div class="line js-file-line" id="LC72">
# Declare OpenNebula templates
</div>
<div class="line js-file-line" id="LC72">
#
</div>
<div class="line js-file-line" id="LC73">
<span class="na">onetemplate</span> <span class="p">{</span> <span class="s">'<name>'</span><span class="p">:</span>
</div>
<div class="line js-file-line" id="LC74">
<span class="na">ensure</span> <span class="o">=></span> <span class="k">present</span> <span class="o">|</span><span class="err"> </span><span class="k">absent</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC75">
<span class="na">memory</span> <span class="o">=></span> <span class="s">'1024'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC76">
<span class="na">cpu</span> <span class="o">=></span> <span class="s">'0.2'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC77">
<span class="na">vcpu</span> <span class="o">=></span> <span class="s">'4'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC78">
<span class="na">os_kernel</span> <span class="o">=></span> <span class="s">'/boot/vmkernel'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC79">
<span class="na">os_initrd</span> <span class="o">=></span> <span class="s">'/boot/vminitrd'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC80">
<span class="na">os_arch</span> <span class="o">=></span> <span class="s">'x86_64'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC81">
<span class="na">os_root</span> <span class="o">=></span> <span class="s">'hda1'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC82">
<span class="na">os_kernel_cmd</span> <span class="o">=></span> <span class="s">'quiet'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC83">
<span class="na">os_bootloader</span> <span class="o">=></span> <span class="s">'/sbin/lilo'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC84">
<span class="na">os_boot</span> <span class="o">=></span> <span class="s">'hd'</span> <span class="o">|</span><span class="err"> </span><span class="s">'fd'</span> <span class="o">|</span><span class="err"> </span><span class="s">'cdrom'</span> <span class="o">|</span> <span class="s">'network'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC85">
<span class="na">acpi</span> <span class="o">=></span> <span class="k">'yes'</span> <span class="o">|</span> 'no'<span class="p">,</span>
</div>
<div class="line js-file-line" id="LC86">
<span class="na">pae</span> <span class="o">=></span> 'yes' <span class="o">|</span><span class="err"> 'no'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC87">
<span class="na">pci_bridge</span> <span class="o">=></span> <span class="s">'4'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC88">
<span class="na">disks</span> <span class="o">=></span> <span class="p">[</span> <span class="s">'disk1'</span><span class="p">,</span> <span class="s">'disk2'</span><span class="p">,</span> <span class="err">...</span><span class="p">],</span>
</div>
<div class="line js-file-line" id="LC89">
<span class="na">nics</span> <span class="o">=></span> <span class="p">[</span> <span class="s">'nic1'</span><span class="p">,</span> <span class="s">'vnet2'</span><span class="p">,</span> <span class="err">..</span> <span class="p">],</span>
</div>
<div class="line js-file-line" id="LC90">
<span class="na">nic_model</span> <span class="o">=></span> <span class="s">'virtio'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC91">
<span class="na">graphics_type</span> <span class="o">=></span> <span class="s">'vnc'</span> <span class="o">|</span> <span class="s">'sdl'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC92">
<span class="na">graphics_listen</span> <span class="o">=></span> <span class="s">'0.0.0.0'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC93">
<span class="na">graphics_port</span> <span class="o">=></span> <span class="s">''</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC94">
<span class="na">graphics_password</span> <span class="o">=></span> <span class="s">'myvncpass'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC95">
<span class="na">graphics_keymap</span> <span class="o">=></span> <span class="s">'de'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC96">
<span class="na">context</span> <span class="o">=></span> <span class="p">{</span> <span class="s">'VAR1'</span> <span class="o">=></span> <span class="s">'value1'</span><span class="p">,</span> <span class="s">'var2'</span> <span class="o">=></span> <span class="s">'value2'</span><span class="p">,</span> <span class="err">...</span><span class="p">},</span>
</div>
<div class="line js-file-line" id="LC97">
<span class="na">context_ssh_pubkey</span> <span class="o">=></span> <span class="s">'$USER[SSH_PUBLIC_KEY]'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC98">
<span class="na">context_network</span> <span class="o">=></span> <span class="s">'yes'</span> <span class="o">|</span> <span class="s">'no'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC99">
<span class="na">context_onegate</span> <span class="o">=></span> <span class="s">'yes'</span> <span class="o">|</span> <span class="s">'no'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC100">
<span class="na">context_files</span> <span class="o">=></span> <span class="p">[</span> <span class="s">'init.sh'</span><span class="p">,</span> <span class="s">'mycontextaddon.sh'</span><span class="p">],</span>
</div>
<div class="line js-file-line" id="LC101">
<span class="na">context_variable</span> <span class="o">=></span><span class="c"> # unused,</span>
</div>
<div class="line js-file-line" id="LC102">
<span class="na">context_placemant_host</span> <span class="o">=></span><span class="c"> # unused,</span>
</div>
<div class="line js-file-line" id="LC103">
<span class="na">context_placement_cluster</span> <span class="o">=></span><span class="c"> # unused,</span>
</div>
<div class="line js-file-line" id="LC104">
<span class="na">context_policy</span> <span class="o">=></span><span class="c"> # unused,</span>
</div>
<div class="line js-file-line" id="LC105">
<span class="p">}</span>
</div>
<div class="line js-file-line" id="LC106">
# Declare OpenNebula VM's
</div>
<div class="line js-file-line" id="LC106">
#
</div>
<div class="line js-file-line" id="LC107">
<span class="na">onevm</span> <span class="p">{</span><span class="err"> </span><span class="s">'<name>'</span><span class="p">:</span>
</div>
<div class="line js-file-line" id="LC108">
<span class="na">ensure </span> <span class="o">=></span> <span class="k">present</span> <span class="o">|</span><span class="err"> </span><span class="k">absent</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC109">
<span class="k">template</span> <span class="o">=></span> <span class="s">'template_name'</span><span class="p">,</span>
</div>
<div class="line js-file-line" id="LC110">
<span class="p">}</span></div>
</blockquote>
<pre></pre>
</div>
Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-76411881612845536952014-07-01T14:24:00.000+02:002014-07-01T14:24:07.074+02:00undefined method parent for nil:NilClassFunny puppet errors:<br />
<br />
err: Could not retrieve catalog from remote server: Error 400 on SERVER: Puppet::Parser::AST::Variable failed with error NoMethodError: undefined method `parent' for nil:NilClass at /etc/puppet/environments/foo/modules/util/manifests/init.pp:23 on node foo.bar.baz<br />
<br />warning: Not using cache on failed catalog<br />err: Could not retrieve catalog; skipping run<br />
<br />
And: it breaks always somewhere different. <br />
<br />
<a name='more'></a><br /><br />
Wow. nice error message.<br />
What does it mean?<br />
The only thing which you can see is that puppet does not receive a class (nil:NilClass)<br />
<br />
It took us some time to find thet we had dangling symlinks in our manifests file structure.<br />
<br />
Puppet definitely has troubles with import (I now hate import even more) and symlinks without a target.<br />
<br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0Berlin, Germany52.520006599999988 13.40495399999997552.210736099999991 12.759506999999974 52.829277099999985 14.050400999999976tag:blogger.com,1999:blog-5525044309410376228.post-60439458807980116402014-05-20T15:49:00.001+02:002023-05-12T09:08:27.122+02:00Puppet rspec testing with automatic hiera lookups (implicit hiera lookup like puppet 3.x)This article is an add-on to my <a href="http://tuxmea.blogspot.de/2013/12/puppet-rspec-testing-and-hiera.html" target="_blank">other article regarding puppet & rspec</a>.<br />
This article only covers automatic hiera lookups used during rspec puppet runs.<br />
Many thanks to Jan (janwaech<at>gmail<dot>com) who spend the time digging into rspec-puppet code finding this nice solution. <br />
<br />
<a name='more'></a><br />
<br />
<h3>
Basic Setup:</h3>
<br />
<h4>
Gemfile</h4>
<br />
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"># Install via<br /># bundle install --path vendor/gems<br />#</span></span></blockquote>
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">source "https://rubygems.org"<br /><br />gem "mocha", :require => false<br />gem 'puppet', '>= 3.1.1'<br />gem 'puppet-lint'<br />gem 'facter', '>= 1.6.10'<br />gem 'rspec-puppet', :git => "https://github.com/rodjek/rspec-puppet.git"<br />gem 'rake', '>= 0.9.2'<br />gem 'puppetlabs_spec_helper', '0.3.0'<br />gem 'test-unit'</span></span></blockquote>
<h4>
Rakefile</h4>
<br />
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">require 'rake'<br />require 'rake/tasklib'<br />require 'rspec/core/rake_task'<br />require 'rubygems'<br />require 'puppetlabs_spec_helper/rake_tasks'</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">require 'hiera'<br />require 'puppet-lint'<br /><br />desc "Run the tests"<br />RSpec::Core::RakeTask.new(:test) do |t|<br /> t.rspec_opts = ['--color', '-f d']<br /> t.pattern = 'spec/*/*_spec.rb'<br />end<br /><br />desc 'Run puppet-lint on the one manifests'<br />task :onelint do<br /> PuppetLint.configuration.with_filename = true<br /><br /> linter = PuppetLint.new<br /> matched_files = FileList['spec/fixtures/modules/one/manifests/**/*.pp']<br /><br /> matched_files.to_a.each do |puppet_file|<br /> linter.file = puppet_file<br /> linter.run<br /> end<br /><br /> fail if linter.errors? || (<br /> linter.warnings? && PuppetLint.configuration.fail_on_warnings<br /> )<br />end<br /><br />task :default => [:spec_prep, :test, :onelint, :spec_clean]</span></span></blockquote>
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> </span></span> <br />
<h4>
spec/spec_helper.rb</h4>
<br />
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">require 'test/unit'<br />require 'mocha/setup'<br />require 'puppetlabs_spec_helper/module_spec_helper'<br /><br />fixture_path = File.expand_path(File.join(__FILE__, '..', 'fixtures'))<br /><br /># include common helpers<br />support_path = File.expand_path(File.join(File.dirname(__FILE__), '..',<br /> 'spec/support/*.rb'))<br />Dir[support_path].each {|f| require f}<br /><br />RSpec.configure do |c|<br /> c.config = '/doesnotexist'<br /> c.manifest_dir = File.join(fixture_path, 'manifests')</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> c.hiera_config = File.join(fixture_path, 'hiera/hiera.yaml')<br /> c.mock_with :mocha<br />end</span></span></blockquote>
<br />
<br />
<h3>
Hiera configuration</h3>
<br />
<h4>
spec/fixtures/hiera/hiera.yaml</h4>
<br />
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">---<br />:backends:<br /> - yaml<br />:hierarchy:</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> - %{::osfamily}<br /> - test<br /><br />:yaml:<br /> :datadir: 'spec/fixtures/hiera' </span></span></blockquote>
<br />
<h4>
spec/fixtures/hiera/test.yaml</h4>
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">---</span></span><span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> </span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">myvariable: 'myvalue'</span></span></blockquote>
<br />
<h4>
spec/fixtures/hiera/Debian.yaml</h4>
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">---</span></span><span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> </span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">myvariable: 'myDebian'</span></span></blockquote>
<br />
<h4>
spec/fixtures/hiera/RedHat.yaml</h4>
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">--- </span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">myvariable: 'myRedHat'</span></span></blockquote>
<br />
<h3>
Hiera integration</h3>
<br />
<h4>
spec/classes/test_spec.rb</h4>
<br />
<blockquote class="tr_bq">
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">require 'spec_helper'<br /><br /># test our main class (init.pp)<br />describe 'stdmodule', :type => :class do # <- set the class name<br /> </span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> context 'on Debian system' do</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> let (:facts) {{ :osfamily => 'Debian' }}<br /> it { should contain_class('stdmodule') } # <- check for classes included<br /> it { should contain_class('stdmodule').with_myparam('myDebian') } # <- check for classes with parameters</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> end</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> context 'on RedHat system' do</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> let (:facts) {{ :osfamily => 'RedHat' }}<br /> it { should contain_class('stdmodule') } # <- check for classes included<br /> it { should contain_class('stdmodule').with_myparam('myRedHat') } # <- check for classes with parameters</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> end</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"></span></span><span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> context 'on Debian system' do</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> let (:facts) {{ :osfamily => 'FooBar' }}<br /> it { should contain_class('stdmodule') } # <- check for classes included<br /> it { should contain_class('stdmodule').with_myparam('myvalue') } # <- check for classes with parameters</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> end</span></span><br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;">end</span></span></blockquote>
<br />
<br />
<span style="font-size: xx-small;"><span style="font-family: Arial,Helvetica,sans-serif;"></span></span>Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-60642476178213749832014-05-08T08:56:00.001+02:002014-05-08T08:57:28.094+02:00Debian on MacBookThere is a very good blog article which only lacks some minor information: http://www.adventuresinoss.com/?p=2516 <br />
<br />
This article adds the required information on how to get grub-efi to work in case of troubles.<br />
<br />
<a name='more'></a><br />
<br />
<br />
After Debian installation re-insert Installer CD/DVD and boot from it.<br />
<br />
Choose the rescue menu and select your root filesystem device (the device where "/" and not where "/boot" is on).<br />
<br />
Check for mounts:<br />
<br />
<blockquote class="tr_bq">
mount -a</blockquote>
<br />
If this command shows you that /boot is mounted try to verify:<br />
<br />
<blockquote class="tr_bq">
umount /dev/<device where /boot is on></blockquote>
<br />
If this gives you an error, than you know for sure that the information was in /etc/mtab but the device was not mounted.<br />
<br />
Now mount your /boot device again:<br />
<br />
<blockquote class="tr_bq">
mount/dev/<device where /boot is on> /boot</blockquote>
<br />
Create efi directory:<br />
<br />
<blockquote class="tr_bq">
mkdir /boot/efi</blockquote>
<br />
Mount your hidden efi partition<br />
<br />
<blockquote class="tr_bq">
mount /dev/<hidden efi partition> /boot/efi</blockquote>
<br />
Load efivars module<br />
<br />
<blockquote class="tr_bq">
modprobe efivars</blockquote>
<br />
Install grub-efi<br />
<br />
<blockquote class="tr_bq">
apt-get install grub-efi</blockquote>
<br />
Rerun update-grub and grub-install<br />
<br />
This will put a new directory in /boot/efi/EFI/ with name debian<br />
<br />
Unmount all partitions, exit the terminal and reboot the system.<br />
<br />
Done.Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-49728076911098952342014-01-16T15:41:00.003+01:002014-09-17T00:07:06.314+02:00Puppet and Hiera HashesHiera data is great for separating code and data.<br />
Within hiera one can put different types of variables:<br />
- text<br />
- arrays<br />
- hashes<br />
<br />
Working with text and arrays is easy for most Puppet users.<br />
<br />
Hashes still seem to have some kind of magic.<br />
This post tries to show you an idea on how to work with hashes in hiera.<br />
<br />
<a name='more'></a><br />
<br />
<br />
<a href="https://www.blogger.com/null" name="more"></a><br />
<br />
<br />
Assumption:<br />
we want ssh host keys in hiera to get added to a system.<br />
<br />
First we are building our hiera data:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">host_key:</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> - server1:</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> alias: webserver</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> key: erupeurzbgpeurhgfpuerfgp</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> type: ssh-rsa</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> - server2:</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> alias: dbserver</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> key: oeurzhgopeurzgbperuzfg</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> type: ssh-dsa</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> - server3:</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> alias: proxy</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> key: 346508365384</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> type: ssh-rsa </span><br />
<br />
One solution is to create a define and use the hash elements within an array.<br />
The define would look like this:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">define process_hostkeys () {</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> ssh_key { $name:</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> alias => $host_key[$name][alias],</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> key => $host_key[$name][key],</span><br />
<span style="font-family: "Courier New", Courier, monospace;"> type => $host_key[$name][type],</span><br />
<span style="font-family: "Courier New", Courier, monospace;">}</span><br />
<br />
So far so good. But... how do we get the elements from the hash?<br />
<br />
Luckily puppet stdlib has the answer: there is a keys() function available.<br />
The keys function extracts all keys from a hash and returns them as an array.<br />
<br />
So in our case this will look like the following:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">$host_array = keys($host_key)</span><br />
<br />
And now we can use our define with this array variable:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">process_hostkeys { $host_array: }</span><br />
<br />
Due to the reason that we may use arrays in titles we are now happy.<br />
<br />
hth.Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0Berlin, Germany52.520006599999988 13.40495399999997552.210736099999991 12.759506999999974 52.829277099999985 14.050400999999976tag:blogger.com,1999:blog-5525044309410376228.post-19868199543366425572013-12-23T11:57:00.003+01:002023-05-12T09:08:54.457+02:00Puppet rspec testing and hieraRspec testing your puppet modules supports you in having stable and functional modules.<br />
For a couple of weeks there is hiera integration in rspec-puppet.<br />
<br />
<br /><a name='more'></a><br />
<h3>
Basic Spec setup</h3>
<br />
You will need a couple of files for puppet module rspec testing:<br />
<br />
Gemfile<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;"># Install via<br /># bundle install --path vendor/gems<br />#</span></span></blockquote>
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;">source "https://rubygems.org"<br /><br />gem "mocha", :require => false<br />gem 'puppet', '>= 3.1.1'<br />gem 'puppet-lint'<br />gem 'facter', '>= 1.6.10'<br />gem 'rspec-puppet', :git => "https://github.com/rodjek/rspec-puppet.git"<br />gem 'rake', '>= 0.9.2'<br />gem 'puppetlabs_spec_helper', '0.3.0'<br />gem 'test-unit'</span></span></blockquote>
Rakefile<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;">require 'rake'<br />require 'rake/tasklib'<br />require 'rspec/core/rake_task'<br />require 'rubygems'<br />require 'puppetlabs_spec_helper/rake_tasks'<br />require 'puppet-lint'<br /><br />desc "Run the tests"<br />RSpec::Core::RakeTask.new(:test) do |t|<br /> t.rspec_opts = ['--color', '-f d']<br /> t.pattern = 'spec/*/*_spec.rb'<br />end<br /><br />desc 'Run puppet-lint on the one manifests'<br />task :onelint do<br /> PuppetLint.configuration.with_filename = true<br /><br /> linter = PuppetLint.new<br /> matched_files = FileList['spec/fixtures/modules/one/manifests/**/*.pp']<br /><br /> matched_files.to_a.each do |puppet_file|<br /> linter.file = puppet_file<br /> linter.run<br /> end<br /><br /> fail if linter.errors? || (<br /> linter.warnings? && PuppetLint.configuration.fail_on_warnings<br /> )<br />end<br /><br />task :default => [:spec_prep, :test, :onelint, :spec_clean]</span></span></blockquote>
spec/spec_helper.rb<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;">require 'test/unit'<br />require 'mocha/setup'<br />require 'puppetlabs_spec_helper/module_spec_helper'<br />require 'puppet/indirector/hiera'<br />require 'hiera'<br /><br /><br />fixture_path = File.expand_path(File.join(__FILE__, '..', 'fixtures'))<br /><br /># include common helpers<br />support_path = File.expand_path(File.join(File.dirname(__FILE__), '..',<br /> 'spec/support/*.rb'))<br />Dir[support_path].each {|f| require f}<br /><br />RSpec.configure do |c|<br /> c.config = '/doesnotexist'<br /> c.manifest_dir = File.join(fixture_path, 'manifests')<br /> c.mock_with :mocha<br />end </span></span></blockquote>
<h3>
Fixtures</h3>
<br />
If your module require other modules (e.g. stdlib functions) you have to include them using fixtures repositories. Add your own module as a symlink:<br />
<br />
.fixtures.yaml<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;">fixtures:<br /> repositories:<br /> stdlib: "https://github.com/puppetlabs/puppetlabs-stdlib.git"<br /> concat: "https://github.com/puppetlabs/puppetlabs-concat.git"</span></span></blockquote>
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;"> symlinks:<br /> <your module name>: "#{source_dir}" </span></span></blockquote>
This will create all required directory structures in spec/fixtures/modules.<br />
<br />
<h3>
Hiera Configuration</h3>
Add a hiera.yaml and a hiera data file to your spec/fixtures:<br />
<br />
spec/fixtures/hiera/hiera.yaml<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;">---<br />:backends:<br /> - yaml<br />:hierarchy:<br /> - test<br /><br />:yaml:<br /> :datadir: 'spec/fixtures/hiera' </span></span></blockquote>
<br />
spec/fixtures/hiera/test.yaml<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;">--- </span></span></blockquote>
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;">myvariable: 'myvalue'</span></span></blockquote>
<br />
<h3>
Hiera Integration</h3>
Only minor changes are required to integrate hiera lookups into your spec tests.<br />
The following example is taken from my stdmodule on github (https://github.com/tuxmea/tuxmea-stdmodule.git)<br />
<br />
spec/classes/<classname>_spec.rb<br />
<blockquote class="tr_bq">
<span style="font-size: x-small;"><span style="font-family: Arial,Helvetica,sans-serif;">require 'spec_helper'<br />hiera_file = 'spec/fixtures/hiera/hiera.yaml' # <- required to find hiera configuration file<br /><br /># test our main class (init.pp)<br />describe 'stdmodule', :type => :class do # <- set the class name<br /><br /> # test without hiera lookup (using class defaults)<br /> context 'without hiera data' do<br /> it { should contain_class('stdmodule') } # <- check for classes included<br /> it { should contain_class('stdmodule').with_myparam('foo') } # <- check for classes with parameters<br /> it { should contain_class('stdmodule::basic') } # <- check for subclasses<br /> it { should contain_class('stdmodule::package_file_service') }<br /> it { should contain_stdmodule__defines('foobar_define') } # <- check for defines<br /> end<br /><br /> # test with explicit params given on class definition<br /> context 'with explicit data' do<br /> let(:params) {{ # <- set explizit params<br /> :myparam => 'newvalue'<br /> }}<br /> it { should contain_class('stdmodule').with_myparam('newvalue') }<br /> end<br /><br /> # test with explicit hiera lookup<br /> context 'with explicit hiera lookup' do<br /> # set hiera_config<br /> let (:hiera_config) { hiera_file }<br /><br /> hiera = Hiera.new(:config => hiera_file) # <- use Hiera ruby class<br /> variable = hiera.lookup('stdmodule::myparam', nil, nil) # <- do hiera lookup<br /> let (:params) {{<br /> :myparam => variable # <- set parameter to hiera lookup<br /> }}<br /> it { should contain_class('stdmodule').with_myparam(variable) }<br /> end<br />end</span></span></blockquote>
Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com1tag:blogger.com,1999:blog-5525044309410376228.post-58672882454194560712013-11-13T16:33:00.000+01:002013-11-13T16:46:33.210+01:00OpenNebula Puppet ModuleWithin my actual project we implemented OpenNebula on CentOS.<br />
The big picture is a self service where developers can startup virtual machines by themselves.<br />
<br />
The
team that worked on the OpenNebula implementation is heavily using
automation for many tasks. Therefore we decided to also automate the
OpenNebula installation and configuration using Puppet.<br />
<br />
<a name='more'></a><br />
<br />
First we were looking for available solution and found the <a href="https://github.com/puppetlabs/puppetlabs-opennebula">puppetlabs OpenNebula module</a> made by Ken Barber.<br />
During PuppetConf 2013 in San Francisco I talked to Ken and he recommended to rewrite the module from scratch.<br />
<br />
Within a couple of days we had a first version of the module finished for internal use only.<br />
(We still had lots of project specific stuff within the module.)<br />
Two weeks later we the module was ready to be made public as Open Source.<br />
<br />
The module can now be found on GitHub:<br />
- <a href="https://github.com/epost-dev/opennebula-puppet-module">ePost Development GitHub account</a><br />
- <a href="https://github.com/tuxmea/opennebula-puppet-module" target="_blank">my fork (this repo has the vtype and ntype available)</a><br />
<br />
The module can be used for a node (compute_node) and the OpenNebula management system (oned).<br />
All configuration is either using defaults (<a href="https://github.com/tuxmea/opennebula-puppet-module/blob/master/manifests/params.pp">manifests/params.pp</a>) or can be fetched from hiera.<br />
<br />
We tried to keep the stuff very modular.<br />
One may choose whether he wants<br />
<ul>
<li>choose virtualization option for a copute node,</li>
<li>choose network option for a compute node,</li>
<li>compute node and/or OpenNebula management,</li>
<li>sunstone or cli only, </li>
<li>apache or webrick as sunstone webserver,</li>
<li>ldap as authentication backend,</li>
<li>sqlite or mysql as data backend.</li>
</ul>
<br />
Module usage inside Puppet:<br />
<br />
1. compute node only (using kvm and 802.1Q):<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "Courier New",Courier,monospace;">class { 'one': }</span></blockquote>
<br />
1.a compute node only (using xen3 and ebtables)<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "Courier New",Courier,monospace;">class { 'one':</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> vtype => 'xen3',</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> ntype => 'ebtables',</span><br />
<span style="font-family: "Courier New",Courier,monospace;">}</span></blockquote>
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<br />
2. OpenNebula management cli only:<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "Courier New",Courier,monospace;">class { 'one':</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> node => false,</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> oned => true,</span><br />
<span style="font-family: "Courier New",Courier,monospace;">}</span></blockquote>
<br />
3. OpenNebula management with Sunstone:<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "Courier New",Courier,monospace;">class { 'one':</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> node => false,</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> oned => true,</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> sunstone => true,</span><br />
<span style="font-family: "Courier New",Courier,monospace;">}</span></blockquote>
<br />
4. Compute Node with OpenNebula Management and Sunstone, using MySQL backend and LDAP authentication:<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "Courier New",Courier,monospace;">class { 'one':</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> node => true,</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> oned => true,</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> sunstone => true,</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> ldap => true,</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> backend => 'mysql',</span><br />
<span style="font-family: "Courier New",Courier,monospace;">}</span></blockquote>
<br />
We are still working on corosync/pacemaker HA setup and we are looking forward to also integrate Oneflow and Onegate.<br />
<br />
Update: add information regarding compute node virtualization and network type. <br />
<br />
<br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-67334289137962297982012-06-27T15:07:00.000+02:002013-11-11T15:01:32.840+01:00Restart puppet using puppetIf you want to ensure puppet upgrade via a puppet manifest you may run into trouble when it comes to puppet service restart.<br /><br />When puppet stops the service it will kill the actual running puppet process.<br />Afterwards puppet does not do the start command.<br /><br /><br /><br /><br/>You can solve this by using the following service resource:<br /><br /><br /><pre>service { 'puppet':<br /> enable => true,<br /> ensure => running,<br /> restart => '/usr/bin/nohup /etc/init.d/puppet restart &',<br />}</pre><br /><br />This will fork a puppet restart command in background.<br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-35335646137075507792012-05-14T19:50:00.000+02:002013-11-11T15:01:32.847+01:00Puppet 2.7.x and Debian ruby 1.9.1According to puppetlabs one should stay with ruby 1.8.7 when running puppet - <a href="http://docs.puppetlabs.com/guides/faq.html#which-versions-of-ruby-does-puppet-support">Puppet FAQ and supported Ruby versions</a>.<br /><br />Most things work, but the CA and SSL has an issue when running ruby 1.9.x - <a href="http://projects.puppetlabs.com/issues/9084">Mixing and matching ruby versions for puppetmasterd and puppetd causes a "certificate verify failed" error</a>.<br /><br />The solution is to create a symlink in the ssl certs directory as described <a href="http://urgetopunt.com/puppet/2011/09/14/puppet-ruby19.html">here</a>.<br /><br/>Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-69538410594152224772012-05-14T13:32:00.000+02:002013-11-11T15:01:32.852+01:00Puppetlabs removes dynamic variable scoping<p>When you run puppet 2.7.12 or higher you will see a warning message, telling you that dynamic variable scope look-up will be removed in puppet 2.8.x.<br /><br />What to do if you have variables defined in a node definition?<br /><br /></p><ol><li>switch to hiera</li><li>write a mockup</li></ol><p></p><br /><br/><br/><p><br />A mockup can be build in the following way:<br /><br /></p><ol><li>write a new module with a define</li><li>use the define and add the proper parameter</li></ol><br />Why do you have to do it that ugly way? Is there no way to give the scope of a node definition?<br /><br />No. Nodes do not have any scope. Scope is limited to modules and classes.<br /><br /><br /><blockquote>class base_variable ( stage = first, dummyvariable ) {<br /> case $dummyvariable {<br /> 'live': { $truevariable = 'live' }<br /> 'testing': { $truevariable = 'testing' }<br /> 'testing2': { $truevariable = 'testing2' }<br /> }<br />}<br /></blockquote><br />In node definition you can now use the define:<br /><br /><blockquote>node 'default.server.domain.tld' {<br /> include stages<br /> class { base_variable:<br /> dummyvariable => 'testing2',<br /> }<br /> include base::dev<br /> ....<br />}<br /></blockquote><br /><p></p><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-57425510993519674542010-08-16T01:09:00.000+02:002013-11-11T15:01:32.858+01:00Debian/GNU Squeeze with Xen 4.0 - Part 1 - base systemThis is the first post regarding Debian/GNU Linux running Xen 4.0 on a new piece of hardware.<br /><br />This posting will cover base OS installation:<br />- Preparation/Partitioning<br />- Base Debian/GNU Linux installation on Dom0<br />- Xen packages installation<br />- Make Xen-Kernel default for Grub2<br /><br />Upcoming posts will cover the following items:<br /><br />- Installation of Debian/GNU Linux as DomU<br />- Installation of Windows Vista as DomU (using HVM)<br />- Installation of OpenSolaris 11 as DomU (using HVM)<br />- Installation of BDS systems as DomU<br /><br /><br /><br/><b>Hardware description</b>:<br /><br />Motherboard: Intel DQ57TM<br />CPU: Intel Core i7<br />RAM: 16 GB RAM<br />HDD: 2x500 GB SATA (internal)<br /><br />Cost (Aug 2010): 1400 Euro<br /><br /><b>Partitioning:</b><br /><br />I want RAID 1 (mirroring) on both disks.<br />I want RAID 1 for / file system and swap<br />I want LVM on RAID 1 for all Virtual Guest systems<br /><br />Trouble:<br /><br />The normal Debian/GNU Linux Lenny (stable) network installation CD does not recognize the network interface (e10000).<br />Base installation required Debian /GNU Linux Squeeze testing network installation cdrom.<br /><br />During Partitioning:<br /><br />- Create empty DOS partition label on both disks<br />- Add RAID Partition for /<br />- Add RAID Partition for swap<br />- Add RAID Partition for LVM<br /><br />/dev/sd[a|b]1 - 15 GB - Linux raid autodetect<br />/dev/sd[a|b]2 - 1 GB - Linux raid autodetect<br />/dev/sd[a|b]3 - 471 GB - Linux raid autodetect<br /><br />Within the Debian installer I choosed to not use any partitions on my empty LVM volume.<br />Logical Volumes will be created after Xen setup.<br /><br />Finish base installation. Do not select any default installation method in taksel.<br /><br /><b>Installation of Xen Kernel</b><br /><br /><br />After first boot into new installed system I installed xen-linux-system-2.6.32-5-xen-amd64:<br /><br /><br /><blockquote><i>apt-get install xen-linux-system-2.6.32-5-xen-amd64</i><br /></blockquote><br /><br />Now I rebooted into Xen Kernel. Since Debian/GNU Linux Version squeeze uses grub2 I reviewed the list of available Kernels and wrote down the number of the Xen Kernel.<br /><br />Then I added "GRUB_DEFAULT=6" into /etc/default/grub and run update-grub afterwards.<br /><br /><b>Network Configuration</b><br /><br />Since I want my Virtual Systems to be available from remote I decided to use Bridging on the network interface.<br /><br />In /etc/network/interfaces I setup eth0 as bridge slave.<br />Then I added a new bridge interface with name xenbr0.<br /><br />In /etc/xen/xend-config.spi I disabled the bridge-network section by giving /bin/true as parameter.<br /><br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com1tag:blogger.com,1999:blog-5525044309410376228.post-8494748312464735962009-03-01T09:53:00.000+01:002013-11-11T15:01:32.865+01:00KDE 4.2 on Debian/GNU LennyI was eagerly waiting for KDE 4.2.<br /><br />Compared to Versoin 4.1 the new version seems far more stable.<br /><br />The only "semi-official" way to install KDE 4.2 on Lenny is by putting experimental in your sources list and make use of apt-pinning.<br /><br />Personally I do not like apt-pinning.<br /><br />Therefor I made a backport.<br /><br /><br/>Put the following into your source.list:<br /><br /> deb http://www.debian-desktop.org/pub/linux/kde42 lenny kde42<br /><br /><br />If you like to review the sources:<br /><br /> deb-src http://www.debian-desktop.org/pub/linux/kde42 lenny kde42<br /><br />The packages are - yet - unsigned.<br />Hopefully I will have some time within the next weeks to also sign the packages.<br /><br /><br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-30143067551220345152009-02-27T15:04:00.000+01:002013-11-11T15:01:32.870+01:00Mac OS X repair disk without boot CDIt will always happen, when you have left your installation CD at home: diskutil is reporting errors which have to be fixed.<br /><br />But: you can not run the repair when you have booted from the disk.<br /><br /><br /><br/><br />restart the system and hold Option+s during boot.<br />This will bring you into single user mode.<br /><br />then run -sbin-fsck -fy and enter exit afterwards.<br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-33816737523707124492008-11-04T15:10:00.000+01:002013-11-11T15:01:32.875+01:00PostgreSQL Replication and Load-Balancing<p>The task was to set up a number of PostgreSQL systems that should always have identical data so one can do load-balancing.<br /><br />I found the tool pgpool-II (http://pgfoundry.org/projects/pgpool/) quite useful for this.<br />Unluckily Debian <span class="caps">GNU</span>/Linux only has Version 1.3 in its repositories (lenny).<br />I took the sources of Version 2.1 and created a new package.<br /><br />Simple load-balancing and replication is quite easy and straight forward. But...<br />what to do if one nodes gets broken or needs the database to be reinitialized?<br /></p><br /><br/><p>Even here pgpool-II offers lots of help by making use of the <span class="caps">PITR </span>(point in time recovery) functionality of PostgreSQL.<br /><br />I came up with the following solution:<br /><br />1. Install first node with postgres and other necessary packages:<br /><br />apt-get install postgresql-8.3 postgresql-client-8.3<br />postgresql-client-common postgresql-common postgresql-contrib-8.3<br /><br />postgresql-server-dev-8.3 make<br /><br />2. create a path for the <span class="caps">PITR </span>archive logs (we need them later for recovery)<br /><br />mkdir /var/lib/postgresql-archive<br />chown postgres. /var/lib/postgresql-archive<br /><br />3. ssh-keys for postgresql user <br /><br />create ssh-keys without passphrase for postgresql user and distribute the key as authorized<em>keys file to all nodes.<br />Hint: we need to copy some data over the filesystem without interaction or login.<br /><br />4. make changes to the postgresql configuration file<br /></em></p><p class="line874"><em>/etc/postgresql/8.3/main/postgresql.conf <span class="anchor" id="line-14"></span><span class="anchor" id="line-15"></span></em></p><p class="line874"><em>listen</em>addresses = '*'</p><p class="line874">archive<em>mode = on <span class="anchor" id="line-18"></span><span class="anchor" id="line-19"></span></em></p><p class="line862"><em>archive</em>command = 'test ! -f /var/lib/postgresql-archive/%f && cp %p /var/lib/postgresql-archive/%f' <span class="anchor" id="line-20"></span><span class="anchor" id="line-21"></span></p>archive_timeout = 60<br /><br />5. make changes to the pgpool configuration file<br /><br /><p class="line874">listen_addresses = '*' <span class="anchor" id="line-28"></span><span class="anchor" id="line-29"></span></p><p class="line874">replication_mode = true <span class="anchor" id="line-30"></span><span class="anchor" id="line-31"></span></p><p class="line874">load_balanced_mode = true <span class="anchor" id="line-32"></span><span class="anchor" id="line-33"></span></p><p class="line874">pgpool2_hostname = 'localhost' <span class="anchor" id="line-34"></span><span class="anchor" id="line-35"></span></p><p class="line874">recovery_timeout = 90 <span class="anchor" id="line-36"></span><span class="anchor" id="line-37"></span></p><p class="line874">replication_timeout = 5000 <span class="anchor" id="line-38"></span><span class="anchor" id="line-39"></span></p><p class="line874">backend_hostname0 = 'postgres-1' <span class="anchor" id="line-40"></span><span class="anchor" id="line-41"></span></p><p class="line874">backend_port0 = 5432 <span class="anchor" id="line-42"></span><span class="anchor" id="line-43"></span></p><p class="line874">backend_weight = 1 <span class="anchor" id="line-44"></span><span class="anchor" id="line-45"></span></p><p class="line874">backend_data_directory0 = '/var/lib/postgresql/8.3/main/' <span class="anchor" id="line-46"></span><span class="anchor" id="line-47"></span></p><p class="line874">backend_hostname1 = 'postgres-2' <span class="anchor" id="line-48"></span><span class="anchor" id="line-49"></span></p><p class="line874">backend_port1 = 5432 <span class="anchor" id="line-50"></span><span class="anchor" id="line-51"></span></p><p class="line874">backend_weight = 1 <span class="anchor" id="line-52"></span><span class="anchor" id="line-53"></span></p><p class="line874">backend_data_directory1 = '/var/lib/postgresql/8.3/main/ <span class="anchor" id="line-54"></span><span class="anchor" id="line-55"></span></p><p class="line874">[...] <span class="anchor" id="line-56"></span><span class="anchor" id="line-57"></span></p><p class="line874">recovery_user = 'postgres' <span class="anchor" id="line-58"></span><span class="anchor" id="line-59"></span></p><p class="line862">recovery_password = <em> </em> <span class="anchor" id="line-60"></span><span class="anchor" id="line-61"></span></p><p class="line867"><em>recovery_1st_stage_command = 'copy-base-backup' </em> <span class="anchor" id="line-62"></span><span class="anchor" id="line-63"></span></p><em>recovery_2nd_stage_command = 'pgpool-recovery-pitr' <br /><br />6. make changes to the configuration file for the pgpool control processor<br /><br />/etc/pcp.conf<br /><br /></em><p class="line874">use the comand pg_md5 to generate md5 hashes of passwords. <span class="anchor" id="line-68"></span><span class="anchor" id="line-69"></span></p><p class="line862">pg_md5 <password> <span class="anchor" id="line-70"></span><span class="anchor" id="line-71"></span></p><p class="line874">add entries to the configuration file: <span class="anchor" id="line-72"></span><span class="anchor" id="line-73"></span></p><username>:<md5 hash of password> <br /><br />7. create 1st stage backup script in /var/lib/postgresql/8.3/main/copy-base-backup<br /><br /><p class="line874">#!/bin/sh <span class="anchor" id="line-79"></span><span class="anchor" id="line-80"></span></p><p class="line874">datadir=$1 DEST=$2 DESTDIR=$3 <span class="anchor" id="line-81"></span><span class="anchor" id="line-82"></span></p><p class="line874"># switch master to prepare for backup <span class="anchor" id="line-83"></span><span class="anchor" id="line-84"></span></p><p class="line874">psql -c "select pg_start_backup('pgpool-recovery')" postgres <span class="anchor" id="line-85"></span><span class="anchor" id="line-86"></span></p><p class="line874"># prepare local command for archive fetching from master warning! scp hostname should be different for every system! <span class="anchor" id="line-87"></span><span class="anchor" id="line-88"></span></p><p class="line862">echo "restore_command = 'scp postgres-1:/var/lib/postgresql-archive/%f %p'" > /var/lib/postgresql/8.3/main/recovery.conf <span class="anchor" id="line-89"></span><span class="anchor" id="line-90"></span></p><p class="line874"># create complete tarball on master <span class="anchor" id="line-91"></span><span class="anchor" id="line-92"></span></p><p class="line874">tar<br />-C /var/lib/postgresql/8.3 -czf main.tar.gz main/global main/base<br />main/pg_multixact main/pg_subtrans main/pg_clog main/pg_xlog<br />main/pg_twophase main/pg_tblspc main/recovery.conf<br />main/backup_label.old <span class="anchor" id="line-93"></span><span class="anchor" id="line-94"></span></p><p class="line874"># switch master back to normal operation <span class="anchor" id="line-95"></span><span class="anchor" id="line-96"></span></p><p class="line874">psql -c 'select pg_stop_backup()' postgres <span class="anchor" id="line-97"></span><span class="anchor" id="line-98"></span></p><p class="line874"># copy tarball to destination <span class="anchor" id="line-99"></span><span class="anchor" id="line-100"></span></p><p class="line862">scp main.tar.gz $DEST:/var/lib/postgresql/8.3/ <span class="anchor" id="line-101"></span><span class="anchor" id="line-102"></span></p># last line <br /><br />8. create 2nd stage recovery script /var/lib/postgresql/8.3/main/pgpool_recovery_pitr<br /><br /><p class="line874">#! /bin/sh <span class="anchor" id="line-106"></span><span class="anchor" id="line-107"></span></p><p class="line874">psql -c 'select pg_switch_xlog()' postgres <span class="anchor" id="line-108"></span><span class="anchor" id="line-109"></span></p># last line <br /><br />9. create post restore initialization script /var/lib/postgresql/8.3/main/pgpool_remote_start<br /><br /><p class="line874">#! /bin/sh <span class="anchor" id="line-113"></span><span class="anchor" id="line-114"></span></p><p class="line874">if [ $# -ne 2 ] <span class="anchor" id="line-115"></span><span class="anchor" id="line-116"></span></p><p class="line874">then <span class="anchor" id="line-117"></span><span class="anchor" id="line-118"></span></p><ul><li style="list-style-type: none;">echo "pgpool_remote_start remote_host remote_datadir" exit 1 <span class="anchor" id="line-119"></span></li></ul><p class="line874">fi <span class="anchor" id="line-120"></span><span class="anchor" id="line-121"></span></p><p class="line874">DEST=$1 <span class="anchor" id="line-122"></span><span class="anchor" id="line-123"></span></p><p class="line874">DESTDIR=$2 <span class="anchor" id="line-124"></span><span class="anchor" id="line-125"></span></p><p class="line874">PGCTL=/usr/lib/postgresql/8.3/bin/pg_ctl <span class="anchor" id="line-126"></span><span class="anchor" id="line-127"></span></p><p class="line862">ssh -T $DEST $PGCTL -w -D /var/lib/postgresql/8.3/main/ stop 2>/dev/null 1>/dev/null < /dev/null <span class="anchor" id="line-128"></span><span class="anchor" id="line-129"></span></p><p class="line874"># delete old content <span class="anchor" id="line-130"></span><span class="anchor" id="line-131"></span></p><p class="line874">ssh<br />-T $DEST 'cd /var/lib/postgresql/8.3/; rm -r main/global main/base<br />main/pg_multixact main/pg_subtrans main/pg_clog main/pg_xlog<br />main/pg_twophase main/pg_tblspc main/recovery.conf<br />main/backup_label.old' <span class="anchor" id="line-132"></span><span class="anchor" id="line-133"></span></p><p class="line874"># expand the archive on the remote system <span class="anchor" id="line-134"></span><span class="anchor" id="line-135"></span></p><p class="line862">ssh -T $DEST 'cd /var/lib/postgresql/8.3/; tar zxf main.tar.gz' 2>/dev/null 1>/dev/null < /dev/null <span class="anchor" id="line-136"></span><span class="anchor" id="line-137"></span></p><p class="line874"># restart postgresql on the remote system <span class="anchor" id="line-138"></span><span class="anchor" id="line-139"></span></p><p class="line862">ssh -T $DEST $PGCTL -w -D /etc/postgresql/8.3/main/ start 2>/dev/null 1>/dev/null < /dev/null & <span class="anchor" id="line-140"></span><span class="anchor" id="line-141"></span></p># last line <br /><br />10. prepare databases<br /><br /><p class="line874">log in to first database node. <span class="anchor" id="line-146"></span><span class="anchor" id="line-147"></span></p><p class="line874">stop postgresql <span class="anchor" id="line-148"></span><span class="anchor" id="line-149"></span></p><p class="line874">cd /var/lib/postgresql/8.3/; tar zxf main.tar.gz main <span class="anchor" id="line-150"></span><span class="anchor" id="line-151"></span></p><p class="line874">stop postgresql on all other nodes <span class="anchor" id="line-152"></span><span class="anchor" id="line-153"></span></p><p class="line874">copy main.tar.gz to all other nodes <span class="anchor" id="line-154"></span><span class="anchor" id="line-155"></span></p><p class="line874">extract main.tar.gz on all other nodes <span class="anchor" id="line-156"></span><span class="anchor" id="line-157"></span></p>start postgresql on all nodes <br /><br />11. install pgpool2 Admin interface (web based)<br /><br /><p class="line862">fetch pgpoolAdmin sources from <a class="http" href="http://pgfoundry.org/projects/pgpool/">http://pgfoundry.org/projects/pgpool/</a> <span class="anchor" id="line-162"></span><span class="anchor" id="line-163"></span></p><p class="line874">apt-get install apache2 libapache2-mod-php5 php5-pgsql <span class="anchor" id="line-164"></span><span class="anchor" id="line-165"></span></p><p class="line874">copy content to /var/www/pgpooladmin <span class="anchor" id="line-166"></span><span class="anchor" id="line-167"></span></p><p class="line862">open a browser: <a class="http" href="http:///">http://</a><nodename>/pgpooladmin/install/ <span class="anchor" id="line-168"></span><span class="anchor" id="line-169"></span></p>follow the setup description (name proper paths to files, set permissions) <br /><br />12. prepare system for pgpool in combination with PITR on line recovery<br /><br /><p class="line874">The postgresql on-line recovery makes use of the PITR (point in time recovery). <span class="anchor" id="line-4"></span><span class="anchor" id="line-5"></span></p><p class="line874">Postgresql writes all transactions to a log (/var/lib/postgresql-archive) and remembers last made transaction. <span class="anchor" id="line-6"></span><span class="anchor" id="line-7"></span></p><p class="line874">To make use of pgpool and on-line recovery the template1 database needs to have the C language function installed. <span class="anchor" id="line-8"></span><span class="anchor" id="line-9"></span></p><p class="line874">cd /root/pgpool2-2.1/sql/pgpool-recovery <span class="anchor" id="line-10"></span><span class="anchor" id="line-11"></span></p><p class="line874">make install <span class="anchor" id="line-12"></span><span class="anchor" id="line-13"></span></p><p class="line874">su - postgres <span class="anchor" id="line-14"></span><span class="anchor" id="line-15"></span></p><p class="line874">cd /root/pgpool2-2.1/sql/pgpool-recovery <span class="anchor" id="line-16"></span><span class="anchor" id="line-17"></span></p><p class="line874">psql -f pgpool-recovery.sql template1 <span class="anchor" id="line-18"></span><span class="anchor" id="line-19"></span></p><br />13. recovery<br /><br />do never run the recovery on the host that you like to recover!!!! <br /><br /><p class="line874">log in to a functional node and run the following commands: <span class="anchor" id="line-27"></span><span class="anchor" id="line-28"></span></p><p class="line862">pcp_detach_node 10 localhost 9898 <username> <password> <number of node to recover> <span class="anchor" id="line-29"></span><span class="anchor" id="line-30"></span></p><p class="line862">pcp_recovery_node 10 localhost 9898 <username> <password> <number of node to recover> <span class="anchor" id="line-31"></span><span class="anchor" id="line-32"></span></p><p class="line874">afterwards log in to the web-admin interfaces on all nodes and enable the recovered system <span class="anchor" id="line-33"></span><span class="anchor" id="line-34"></span></p>or log in to all other nodes and run <br /><br />pcp_attach_node 10 localhost <username> <password> <number of node to attach> <br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-398967779613705912008-08-17T00:30:00.000+02:002013-11-11T15:01:32.884+01:00nagios redundant master servers setupIt may be necessary to have a fail-over nagios master in case of primary system hardware failure.<br />The following describes the setup of nagios redundant master setup using heartbeat and nagios internal techniques.<br /><br />Both nagios systems have to be set up identically in terms of monitoring items.<br />Nagios will be running as process on both cluster nodes.<br /><br />After finishing setup the following is needed:<br /><br />- both systems need to have passive checks enabled<br />- both systems need nsca running<br />- both systems will need the obsess_over_services option set to 1<br />- both systems will need an ocsp_command configured (submit_check_result)<br /><br />The command submit_check_results needs to be configured:<br /><br /><blockquote><i>define command{</i><br /><i> command_name submit_check_result</i><br /><i> command_line /usr/lib/nagios/libexec/eventhandlers/submit_check_result $HOSTNAME$ '$SERVICEDESC$' $SERVICESTATE$ '$SERVICEOUTPUT$'</i><br /><i> }</i><br /></blockquote><br />The mentioned script needs to be put into place on the primary node: <br /><span class="mt-enclosure mt-enclosure-file" style="display: inline;"><a href="http://www.unix-freak.org/2008/08/16/monitor1">submit_check_result</a></span><br /><br />Now we would need an additional start-stop script: <br /> /etc/init.d/nagios_notification<br /><br /><blockquote><i>#!/bin/bash</i><br /><br /><i>case "$1" in</i><br /><i> "start")</i><br /><i> echo "Starting notifications"</i><br /><i> /usr/lib/nagios/libexec/eventhandlers/enable_notifications</i><br /><i> echo "Done"</i><br /><i> exit 0</i><br /><i> ;;</i><br /><i> "stop")</i><br /><i> echo "Stopping notifications"</i><br /><i> /usr/lib/nagios/libexec/eventhandlers/disable_notifications</i><br /><i> echo "Done"</i><br /><i> exit 0</i><br /><i> ;;</i><br /><i> *)</i><br /><i> echo "Usage: $0 [start|stop]"</i><br /><i> ;;</i><br /><i>esac</i><br /></blockquote><br /><br /><br />Now we can start with the heartbeat setup:<br /><br />put into /etc/ha.d/haressources on both clusternodes:<br /><br /><blockquote><i><clustername> \</i><br /><i> IPaddr::<virt ip>/cidr>/<iface>/<bcast> \</i><br /><i> nagios_notifications</i><br /></blockquote> et voila.<br /><br />We now have a basic nagios cluster where node B is informed about updates that are done on node A.<br />In case of hardware failure node B will take over and have notifications enabled.<br /><br />Best would be to also disable active checks on node B until fail-over.<br /><br />I will add an update on this.<br /><br /><br /><br/>It may be necessary to have a fail-over nagios master in case of primary system hardware failure.<br />The following describes the setup of nagios redundant master setup using heartbeat and nagios internal techniques.<br /><br />Both nagios systems have to be set up identically in terms of monitoring items.<br />Nagios will be running as process on both cluster nodes.<br /><br />After finishing setup the following is needed:<br /><br />- both systems need to have passive checks enabled<br />- both systems need nsca running<br />- both systems will need the obsess_over_services option set to 1<br />- both systems will need an ocsp_command configured (submit_check_result)<br /><br />The command submit_check_results needs to be configured:<br /><br /><blockquote><i>define command{</i><br /><i> command_name submit_check_result</i><br /><i> command_line /usr/lib/nagios/libexec/eventhandlers/submit_check_resul</i><br /><i>t $HOSTNAME$ '$SERVICEDESC$' $SERVICESTATE$ '$SERVICEOUTPUT$'</i><br /><i> }</i><br /></blockquote><br />The mentioned script needs to be put into place: <br /><span class="mt-enclosure mt-enclosure-file" style="display: inline;"><a href="http://www.unix-freak.org/2008/08/16/monitor1">monitor1</a></span><br /><br />Now we would need an additional start-stop script: <br /><br />Now we can start with the heartbeat setup:<br /><br />put into /etc/ha.d/haressources:<br /><br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-21239396840175853122008-08-16T20:47:00.000+02:002013-11-11T15:01:32.892+01:00example for munin-nagios integrationLet's assume that you have an USB temperature sensor connected to a machine in your server room.<br />Active checks would make no sense here, since most of the time the temperature should be OK.<br />But you want to get notified in case of cooling problems and over heating.<br /><br /><br /><br/><br />First you need a plugin for munin:<br /><span class="mt-enclosure mt-enclosure-file" style="display: inline;"><a href="http://www.unix-freak.org/2008/08/16/serverroom">serverroom</a></span><br /><br /><br /><br />Verifiy that this plugin is working prior doing anything else!<br />Also adopt the warn and crit values to your needs.<br /><br />Now you need nsca installed on your munin master node.<br /><br />Next<br />step is a proper munin master configuration for your system that has<br />the USB thermometer connected. We assume that the system has the name<br />intranet and that you have configured munin to make use of some<br />domains. intranet is located in domain intern.<br /><br /><blockquote><i>contact.nagios.command /usr/sbin/send_nsca -H <nagios server IP> -c /etc/send_nsca.cfg</i><br /><i>[intranet.intern]</i><br /><i> notify_alias intranet</i><br /><i> address <your systems IP></i><br /><i> use_node_name yes</i><br /></blockquote><br />If you omit the "notify_alias" part all alarms will be sent from the given system name plus domain appended (intranet.intern).<br />With the notify alias you can make sure that nagios receives the alarm for the proper system.<br /><br />Now the nagios system needs to get configured.<br />Make sure you have nsca installed and running.<br />Now enable passive chacks in nagios.conf and create a service for the temperature alarm:<br /><br /><blockquote><i>define service{</i><br /><i> use passive-service</i><br /><i> hostgroup_name temperature-servers</i><br /><i> service_description Serverroom</i><br /><i> check_command return-ok</i><br /><i> }</i><br /></blockquote><br /><br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-17764852236004749982008-08-01T01:14:00.000+02:002013-11-11T15:01:32.902+01:00combining nagios and munin<a href="http://munin.projects.linpro.no/">munin</a> offers a nice way to collect and show system information. So munin can be easily used as a monitoring system. Munin uses simple plugins (either shell- or perl code) to gather data from systems.<br /><a href="http://www.nagios.org/">nagios</a> is well-known as an alarming system.<br /><br />munin offers the possibility to name warn and critical values. In single use munin will show items that are beyond their warn and critical values by link highliting.<br /> <br /><div>Additionally munin offers the possibility to make use of nagios passive checks via nsca.<br /><br /></div><br /><br/>nsca is a part of nagios.<br /><br />first one needs to configure the munin-master to make use of the send_nsca command.<br />second one has to configure the nagios master to also run the nsca daemon (either via inetd or as standalone daemon) <br /><br />Since nsca documentation is very simple the guys from munin made <a href="http://munin.projects.linpro.no/wiki/HowToContactNagios">a documentation</a> on how to combine munin and nagios.<br /><br />The advantage of this is that you get information upon changes immediately.<br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com1tag:blogger.com,1999:blog-5525044309410376228.post-72902534869310844392008-07-29T23:07:00.000+02:002013-11-11T15:01:32.912+01:00Nagios - different alarm schemes for different systemsMost companies have live and development systems.<br />Problems on live systems should be made known to the sysadmin immediately.<br />Problems on development systems should not cause an email to be sent immediately.<br /><br />Nagios offers possibilities to have different alarming schemes for different hosts.<br /><br />1. setup a new contact group<br />2. setup a new generic service<br />3. setup two hostgroups (one for live servers, one for development systems.)<br />4. setup services to monitor for each hostgroup. make use of the earlier defined service.<br /><br /><br/>define contacts:<br /><br />define contact{<br /> contact_name live-root<br /> alias Root live<br /> service_notification_period 24x7<br /> host_notification_period 24x7<br /> service_notification_options w,u,c,r<br /> host_notification_options d,r<br /> service_notification_commands notify-service-by-email<br /> host_notification_commands notify-host-by-email<br /> email live-root@localhost<br /> }<br /><br />define contact{<br /> contact_name devel-root<br /> alias Root devel<br /> service_notification_period 24x7<br /> host_notification_period 24x7<br /> service_notification_options w,u,c,r<br /> host_notification_options d,r<br /> service_notification_commands notify-service-by-email<br /> host_notification_commands notify-host-by-email<br /> email devel-root@localhost<br /> }<br /><br />define contactgroup<br /><br />define contactgroup{<br /> contactgroup_name live-admins<br /> alias Nagios live Administrators<br /> members live-root<br /> }<br /><br />define contactgroup{<br /> contactgroup_name devel-admins<br /> alias Nagios devel Administrators<br /> members devel-root<br /> }<br /><br /><br />define new generic service<br /><br />define service{<br /> name live-service<br /> active_checks_enabled 1<br /> passive_checks_enabled 1<br /> parallelize_check 1<br /> obsess_over_service 1<br /> check_freshness 0<br /> notifications_enabled 1<br /> event_handler_enabled 1<br /> flap_detection_enabled 1<br /> failure_prediction_enabled 1<br /> process_perf_data 1<br /> retain_status_information 1<br /> retain_nonstatus_information 1<br /> notification_interval 0<br /> is_volatile 0<br /> check_period 24x7<br /> normal_check_interval 5<br /> retry_check_interval 1<br /> max_check_attempts 4<br /> notification_period 24x7<br /> notification_options w,u,c,r<br /> contact_groups live-admins<br /> register 0<br /> }<br /><br />define service{<br /> name devel-service<br /> active_checks_enabled 1<br /> passive_checks_enabled 1<br /> parallelize_check 1<br /> obsess_over_service 1<br /> check_freshness 0<br /> notifications_enabled 1<br /> event_handler_enabled 1<br /> flap_detection_enabled 1<br /> failure_prediction_enabled 1<br /> process_perf_data 1<br /> retain_status_information 1<br /> retain_nonstatus_information 1<br /> notification_interval 0<br /> is_volatile 0<br /> check_period 24x7<br /> normal_check_interval 5<br /> retry_check_interval 1<br /> max_check_attempts 4<br /> notification_period workhours # defined in timeperiods<br /> notification_options w,u,c,r<br /> contact_groups devel-admins<br /> register 0<br /> }<br /><br />define hostgroups<br /><br />define hostgroup {<br /> hostgroup_name live-servers<br /> alias Live Systems<br /> members live-server-1<br /> }<br /><br />define hostgroup {<br /><br /> hostgroup_name devel-servers<br /><br /> alias Devel Systems<br /><br /> members devel-server-1<br /><br /> }<br /><br />define service<br /><br />define service {<br /> hostgroup_name live-servers<br /> service_description PING<br /> check_command check_ping!100.0,20%!500.0,60%<br /> use generic-service<br /> notification_interval 0 ; set > 0 if you want to be renotified<br />}<br /><br />define service {<br /> hostgroup_name devel-servers<br /> service_description PING<br /> check_command check_ping!100.0,20%!500.0,60%<br /> use generic-service<br /> notification_interval 0 ; set > 0 if you want to be renotified<br />}<br /><br /><br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-17844728983089721252008-07-29T00:52:00.000+02:002013-11-11T15:01:32.921+01:00puppet and key managementIn case that you need to set up a system that has already benn managed by puppet one does need to run some additional steps.<br />First you need to remove the key from puppetmaster:<br /><br />puppetca --clean <hostname><br /><br />Then you may set up the old system from scratch,.<br /><br />After puppet startup use the puppetca command on puppet master to look and sign the new key<br /><br /><br/>In case that puppetca --list will not show the new host key you need to run the following steps:<br /><br />1. remove ssl-keys from puppet client<br />2. start puppet on client<br />3. run puppetca --list on puppetmaster.<br /><br /><br />Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0tag:blogger.com,1999:blog-5525044309410376228.post-36478920008479447962008-07-26T21:42:00.000+02:002013-11-11T15:01:32.926+01:00puppet automated system configurationWe use <a href="http://reductivelabs.com/">puppet</a> for automated system configuration for some time.<br />The developers are naming puppet the successor of cfengine.<br />Puppet is written in ruby and supports the following platforms:<br />- Linux<br />- OS X<br />- BSD<br />- Solaris<br /><br />I will add some notes of puppet - especially about items that took us some time to find out - in the near future.<br /><br /><br/>Martin Alfkehttp://www.blogger.com/profile/12841112135771278690noreply@blogger.com0