munin 1.4 on debian 5

Due to the fact that in the stable debian the munin version is 1.2.x, which is to old for me, I have build now the latest stable munin version 1.4.x for stable debain.

Here now the steps how I have reached the munin_1.4.5-2_all.deb, munin-common_1.4.5-2_all.deb, munin-java-plugins_1.4.5-2_all.deb, munin-node_1.4.5-2_all.deb, munin-plugins-extra_1.4.5-2_all.deb for installation

aptitude install svn-buildpackage quilt htmldoc sun-java6-jdk dpkg-dev

dpkg-dev is for dpkg-buildpackage

svn co svn://munin.projects.linpro.no/munin/branches/1.4-stable/ 1.4-stable_20100810
cd 1.4-stable_20100810
svn co svn://munin.projects.linpro.no/munin/branches/debian/squeeze/trunk/debian debian

change one line in debian/Makefile.config

vim debian/Makefile.config
JC         = /usr/lib/jvm/default-java/bin/javac
to
JC         = /etc/alternatives/javac

and in debian/control

vim debian/control
Build-Depends-Indep: perl, htmldoc, html2text, default-jdk
Build-Depends-Indep: perl, htmldoc, html2text, sun-java6-jdk

After all this steps you can execute

dpkg-buildpackage

as described in the README

On every node you must also install the following packages, at least.

aptitude install gawk rrdtool librrds-perl liblog-log4perl-perl libnet-server-perl

now you can install the packages as usual with dpkg -i

UPDATE: I opened a ticket

persistent memcache

From time to time  I search for the possibility to make memcache persistent and  high available.

I found the Tokyo Tyrant server which solves this requirement.

This server is written from Mikio Hirabayashi, on his homepage you can find some other nice tool ;-)

FPM in php 5.3.3

The FastCGI Process Manager (FPM) is now part of the official php 5.3 branch.

http://www.php.net/archive/2010.php#id2010-07-22-2

The  official site in the php manual can be found here

The official site for the php-fpm project is here

Build Alfresco community edition

Due to a bug in alfresco 3.3 I decided to build my own alfresco community edtion.

My first step was to download the latest tomcat 6 version and extract the tarball

cd /datadir/al/download/
mkdir alfresco
cd !$
mv <DOWNLOADED_.tar.gz> .
tar xfvz apache-tomcat-6.0.28.tar.gz

the second step was to read Developer Runtime Configuration.

Then get the  current subversion HEAD, change to the checkedout directory and set some env vars.

svn co svn://svn.alfresco.com/alfresco/HEAD alfresco_20100711
cd alfresco_20100711/root
export TOMCAT_HOME=/datadir/al/download/alfresco/apache-tomcat-6.0.28
export VIRTUAL_TOMCAT_HOME=/datadir/al/download/alfresco/apache-tomcat-6.0.28
export APP_TOMCAT_HOME=/datadir/al/download/alfresco/apache-tomcat-6.0.28/
export CATALINA_OPTS="-server -Xmx1024M -XX:PermSize=128m"
ant -verbose -l ../ant-20100713-2303.log

on my machine was the buildtime ~30 minutes.

Now you have a own alfresco.

Before you start the tomcat extract the original server.xml and web.xml from the tomcat tarball.

cd ../..
tar xfvz apache-tomcat-6.0.28.tar.gz  apache-tomcat-6.0.28/conf/server.xml apache-tomcat-6.0.28/conf/web.xml
cd apache-tomcat-6.0.28
bin/startup.sh

after the successfull startup can you connect to the alfresco share.

http://localhost:8080/share/

If you have some suggestion or optimizations please drop me comment ;-)

TODOs:

  1. create a empty alf_data. I have used the alf_data from a installed 3.3
  2. add alfresco-global.properties to the alfresco.war. Current solution: cp /home/al/Alfresco/tomcat/shared/classes/alfresco-global.properties /datadir/al/download/alfresco/apache-tomcat-6.0.28/webapps/alfresco/WEB-INF/classes/

Listen and save a live stream

I have tried to save some music from the local radio stations Ö3 and Kronehit.

After some try and errors I came to this both lines

Ö3

gst-launch-0.10 -tv uridecodebin subtitle-encoding=UTF-8 \
uri=mms://gcssrv.pkf.speednet.at/WSX/oe3_live \
! tee name=myT myT. ! queue2 \
! alsasink myT. ! queue2 ! audioconvert! lamemp3enc \
! multifilesink location=oe3_%05d

Kronehit

gst-launch-0.10 -tv uridecodebin subtitle-encoding=UTF-8 \
uri=http://onair.krone.at/kronehit.mp3 \
! tee name=myT myT. ! queue \
! alsasink myT. ! queue ! audioconvert! lamemp3enc \
! multifilesink location=kronehit_%05d

The next step would be to save the stream automatical with the right name => Interpreter_Song.

If you have some ideas how I can do this please drop me a message, thanks ;-)

evolution on ubuntu 9.10

I have tried to use the latest evolution sources, as described in

http://mad-scientist.us/evolution.html => how to use it

before you can start to build the evolution  you will need to install the following packages

sudo aptitude install libnspr4-dev libnss3-dev ccache evolution-dev git-core gperf gtk-doc-tools libcanberra-gtk-dev libdb-dev libdbus-glib-1-dev libdcerpc-dev libffi-dev libgnome-desktop-dev libgstreamer0.10-dev libgtkhtml-editor-dev libgtkimageview-dev libhal-dev libical-dev libkrb5-dev libldap2-dev libmapi-dev libparse-pidl-perl libproxy-dev libpst-dev libsamba-hostconfig-dev libtalloc-dev libtiff4-dev libtorture-dev libunique-dev network-manager-dev python-dev tdb-dev

You will not be able to build due to the follwoing error.

http://packages.ubuntu.com/search?searchon=contents&keywords=introspection.m4&mode=filename&suite=karmic&arch=any

Which means no introspection.m4

If anybody is able to help to fix this issue => Thank you for your help ;-)

play radio stream with gstreamer

Due to the fact that I started to play with gstreamer I have needed audio video inputs.

as we know the best source for such content is the NET I just search for some radio streams and found two Austrian local senders.

Ö3 http://oe3.orf.at/ => mms://gcssrv.pkf.speednet.at/WSX/oe3_live

88.6 http://886.at/   => http://radiostream.de/stream/36795.pls

For  Ö3 it is easy,  just use on gentoo:

gst-launch-0.10 -v playbin2 uri=mms://gcssrv.pkf.speednet.at/WSX/oe3_live

If you have another distribution maybe it is enough gst-launch.

For the 88. 6 is it a little bit difficulty, due to the fact that the link is a multimedia playlist.

  1. get the content of the pls file
curl -s http://radiostream.de/stream/36795.pls|egrep File
  1. take one of the streams and handover it to the gstreamer
gst-launch-0.10 -v playbin2 uri=http://ber.radiostream.de:36795

That’s it, have fun ;-)

VNC and SSH for portforwarding

The problem: There is only a Windows PC with a Internet connection and you need to connect to some unix machines behind this machine with ssh, HOW?

The solution: Use a VNC (e.g.: TightVNC) and OpenSSHD for Windows on your Linux box.

Detail description:

Pre required components:

  1. A VNC Client, I have used the vncviewer from TightVNC
  2. OpenSSHD for Windows
  3. Admin access to the Windows machine

OK now we can start with the necessary steps.

Windows Part

1.) Install TightVNC on your Windows machine.

2.) Install OpenSSHD on your Windows machine.

3.) Read the quickstart.txt from OpenSSHD.

Here the part which I think ist the most important one:

Configuration
1.  Open a command prompt and change to the installation directory
    (Program Files\OpenSSH is the default).
2.  CD into the bin directory.
3.  Use mkgroup to create a group permissions file. For local
    groups, use the "-l" switch. For domain groups, use the
    "-d" switch.
    For both domain and local, it is best to run the command twice
    (remember to use >>, not >). If you use both, make sure to edit
    the file to remove any duplicate entires.
    mkgroup -l >> ..\etc\group      (local groups)
    mkgroup -d >> ..\etc\group      (domain groups)
4.  Use mkpasswd to add authorized users into the passwd file. For
    local users, use the "-l" switch. For domain users, use the
    "-d" switch.
    For both domain and local, it is best to run the command twice
    (remember to use >>, not >). If you use both, make sure to edit
    the file to remove any duplicate entires.

    mkpasswd -l [-u <username>] >> ..\etc\passwd      (local users)
    mkpasswd -d [-u <username>] >> ..\etc\passwd      (domain users)

    NOTE: To add users from a domain that is not the primary domain
          of the machine, add the domain name after the user name.
    NOTE: Ommitting the username switch adds ALL users from the
          machine or domain, including service accounts and the
          Guest account.

5.  Start the OpenSSH server.

       net start opensshd

6.  Test the server. Using a seperate machine as the client is best.
    If you connect but the connection immediately gets dropped,
    reboot the machine with the server and try connecting again.

Pay attention that you have a Windows user with a password. This password will be used for the SSH connection.

4.) Start TightVNC and go to the Properties.

5.) Switch to the Administration tab and activate the loopback connections

tightvnc options

6.) Now we are done on the Windows PC

Linux Part

1.) Install TightVNC on your unix / linux machine.

2.) Start a shell

3.) Type in the target server as SECOND -L

(this must be in one line!)

export VNC_VIA_CMD=’/usr/bin/ssh -f -L %L:%H:%R -L 4711:<DESTINATION_SERVER_IN_THE_REMOTE_LAN>:22 <WINDOWS_USERNAME>@%G sleep 20′

4.) Connect to the Windows connected PC with vncviewer

(this must be in one line!)

echo <YOUR_TIGHT_VNC_PW>|vncviewer -compresslevel 4 -quality 4 -bgr233 -autopass -via <IP_OF_THE_INTERNET_CONNECTED_PC> <IP_OF_THE_INTERNET_CONNECTED_PC>

e.g.:

echo aaa|vncviewer -compresslevel 4 -quality 4 -bgr233 -autopass -via 192.168.1.41 192.168.1.41

5.) Now you can connect via port 4711 to the <DESTINATION_SERVER_IN_THE_REMOTE_LAN>

ssh -p 4711 127.0.0.1

EH Voila we are now connected to the remote site ;-)
Here a picture because its easier to understand ;-)

Connection Flow

Country select box in symfony setWidgets

For my register form was the requirement to select from which Country a user is.

Symfony have a nice feature called select_country_tag which I wanted to use in the $this->setWidgets() in the mysfGuardFormSignin.class.php the problem is that the select_country_tag gives not the array back.

After a little bit ‘reverse engineering’ the solution for me is like this

    $c = new sfCultureInfo(sfContext::getInstance()->getUser()->getCulture());
    $countries = $c->getCountries();

here the full class

less lib/form/mysfGuardFormSignin.class.php
<?php

class mysfGuardFormSignin extends sfForm
{
  public function configure()
  {
    $c = new sfCultureInfo(sfContext::getInstance()->getUser()->getCulture());
    $countries = $c->getCountries();

    $this->setWidgets(array(
      'username' => new sfWidgetFormInput(),
      'email' => new sfWidgetFormInput(),
      'password' => new sfWidgetFormInput(array('type' => 'password')),
      'country'  => new sfWidgetFormSelect(array('choices' => $countries)),
    ));

    $this->setValidators(array(
      'username' => new sfValidatorString(),
      'email'    => new sfValidatorString(),
      'password' => new sfValidatorString(),
      'country'  => new sfValidatorString(),
    ));

    $this->validatorSchema->setPostValidator(new sfGuardValidatorUser());

    $this->widgetSchema->setNameFormat('signin[%s]');
  }
}

Which language for Webdevelopment php python perl ruby or java

I have searching for a long time for the right webframewor to develop a rather small Webapplication.
The requirement was:

  1. Usermanagmemtn, ACL
  2. Manage Files to download by Userpermission
  3. run behind nginx

I have developed in all of this languages in the past, some of them in mor deep then in the other.

I have take a look about different frameworks in this languages

  1. Perl
  2. Python
  3. PHP
  4. Ruby
  5. Java

The main decision to go to the php based symfony framework was:

  1. It is easy to learn and relativly good documented
  2. php-fm
  3. I know php more or less good, not better then perl but the setup was easier to create.
  4. There are more php known persons in the current company.

My reasons why I have not choose one of the differnt languages to work:

  1. Java: due the fact that you need to compile it, so you need a IDE (ala. Eclipse). It takes more time to see if you change works.
  2. ruby & python: I haven’t not enough experience with it. This language looks very nice and easy to learn but it wasn’t necessary that I learn it, yet ;-)
  3. perl: I have develop a lot of perl modules and code in the past but for now I like php more.

So as you can see it is not very academic more pragamtic ;-) .

High performance PHP

If you have the need to run php with a very high traffic you should think about the follwoing setup.

Don’t think it will work in 5 minutes and you don’t need some knowledge about how to setup an webserver and a php application, you definitely need it.

If you think this is a step by step description for people which only want to copy and past commands and it works then you are wrong here, please take a websearch engine for a alternative description.

What you need:

  1. nginx http://www.nginx.net/
  2. php-fm http://php-fpm.org/
  3. php http://www.php.net/
  4. a compile environment
  5. the development libraries for the wanted php features and libxml2
  6. A clear test scenario and target how many requests per second should be reached on static and dynamic part.

How you can build this environment:

  1. Extract, build and install nginx http://wiki.nginx.org/NginxGettingStarted
  2. Extract, build and install php-fm http://php-fpm.org/wiki/Documentation
  3. configure nginx for fast-cgi proxying http://wiki.nginx.org/NginxHttpFcgiModule

High Performance View

A brief description how the flow is:

  • A request comes to nginx
  • He takes a look into his config to find the right location or server config
  • If the static content rule matches then he will deliver the content directly from the disc
  • If the dynamic rule matches the he will contact php-fm, which manage the php-instances
  • php-fm execute the php script and send the dynamic content back to nginx
  • nginx sends the content back to the client

There are some pitfalls:

  1. client_body_buffer_size
  2. fastcgi_buffers*

You should take your time to find the right config setup for nginx and php-fm to be able to fulfill the assumed traffic.

I have started to translate the russian documentation, which is still not complete.

UPDATE: There is now a official wiki http://php-fpm.org/

use the event ports in nginx

Solaris have since version 10 a new polling infrastrucutre Event Ports here a more detailed article about Event Completion Framework. Adam Leventhal wrotes in his Blog for him this is one of the top 20 best thing in Solaris 10.

By default nginx do not use this event methode due to the fact that there was some security related problems Security Vulnerabilities in The Solaris Event Port API May Result in a Denial of Service (DoS) Condition and  Vulnerability in the Solaris 10 Event Port Implementation May Lead to a System Panic, Resulting in a Denial of Service (DoS) if you have a newer Version or applied the Patch you should use the event ports in nginx.

You only need to add

use eventport;

in the events { … } section. If you don’t add this then nginx by default us the /dev/poll method.

More about the possible polling methods can be found here.

haproxy on Opensolaris 2008.05

When you want to build haproxy on Opensolaris 2008.05 you need the gcc and the gmake due to fact that the Sun CC does not support fully the C99 standard and the Makefile use GNU Make features.

I have tried to build haproxy with -xc99 but it was unable due the following code in src/client.c

/* Note: must not be declared <const> as its list will be overwritten */
static struct acl_kw_list acl_kws = {{ },{
{ “src_port”, acl_parse_int, acl_fetch_sport, acl_match_int },
{ “src”, acl_parse_ip, acl_fetch_src, acl_match_ip },
{ “dst”, acl_parse_ip, acl_fetch_dst, acl_match_ip },
{ “dst_port”, acl_parse_int, acl_fetch_dport, acl_match_int },
#if 0
{ “src_limit”, acl_parse_int, acl_fetch_sconn, acl_match_int },
#endif
{ “dst_conn”, acl_parse_int, acl_fetch_dconn, acl_match_int },
{ NULL, NULL, NULL, NULL },
}};

and some other code. I think SUN should make his cc compliant so that we are able to build haproxy with there compiler ;-) . When you install the gcc and add the follwoing statement into the Makefile

-I$(PCREDIR)/include/pcre

in both *_PCRE sections then you are able to compile haproxy with the follwoing options

gmake -f Makefile TARGET=solaris USE_STATIC_PCRE=1 CPU=i686

for the final build you must rearrange the link line, this line works for me

gcc -g -o haproxy src/haproxy.o src/sessionhash.o src/base64.o src/protocols.o src/uri_auth.o src/standard.o src/buffers.o src/log.o src/task.o src/time.o src/fd.o src/regex.o src/cfgparse.o src/server.o src/checks.o src/queue.o src/client.o src/proxy.o src/proto_uxst.o src/proto_http.o src/stream_sock.o src/appsession.o src/backend.o src/senddata.o src/dumpstats.o src/proto_tcp.o src/session.o src/hdr_idx.o src/ev_select.o src/acl.o src/memory.o src/ebtree.o src/eb32tree.o src/ev_poll.o -L/usr/lib -Wl,-Bstatic -Wl,-Bdynamic -lnsl -lsocket -lpcreposix -lpcre

The point was, which looks to me, that the libs must be added as last options.

nginx build on opensolaris with SunStudio

Today I was interested if the latest nginx (nginx-0.7.1) is able to build on the opensolaris box and YES it does ;-)

1.) get it

wget http://sysoev.ru/nginx/nginx-0.7.1.tar.gz

2.) extract it

gtar xfvz nginx-0.7.1.tar.gz

3.) build it

cd nginx-0.7.1 && ./configure –prefix=$HOME/servers/nginx –with-http_ssl_module –with-debug –with-cc=/opt/SunStudioExpress/bin/cc

It’s so easy, thanks Igor.

Open Solaris 2008.05

Today I got the Opensolaris installation over the Virtualbox.

It looks much nicer with gnome but since Solaris 9 and 10 there was a lot of differences like nwamd and some other ‘typical’ Systemadmin commands.

I’am curious about the new features and handling ;-)

My interests

I like to work with nginx and haproxy and all of the high performance small resources issue.

You can find me in xing or post a message to contact me.

First Entry

Hi all,

this is the BLOG of Aleksandar Lazic.
I hope this will not be ‘yust another blog’ but we well see in the future ;-)

Follow

Get every new post delivered to your Inbox.