Jesień Linuksowa 2018

Last weekend I participated in the conference Jesień Linuksowa 2018 in Ustroń which is located in Poland near the Czech and Slovak borders. It was my first time in a country with so much tragic historical experiences.

On the other hand, I was impressed by the community members and the organization of the event. We celebrated another edition of Linux Autumn in the hotel Gwarek and my post-event wrap up will take into consideration seven basic points:

Organizers

This time I was accompanied by my friend Ana Garcia, who is a student at the University of Edinburgh and the members of the organization were supportive and kind all the time with us. We felt a warm environment since we arrive at midnight in the middle of the fog. They helped us with our talks and workshops we offer related to parallelization.

We meet new Linux friends: Dominik, Rafal, Filip, Mateusz Kita, and Matej from Red Hat.Topics

Interesting topics were presented that included Ansible, catching bugs, packaging to Fedora and the innovations of systemd. Here pictured Marcin, Matej, and Zbigniew talks:

My participation

I did a topic related to introduction to parallel computers and a workshop to use parallel directives in OpenMP such as static, dynamic and guided. Thanks to Ana for being my co-speaker this time. My talk was set to the morning around 10 am, and we did the workshop after lunch and we are glad that all the participants completed successfully the experience.

Fedora 29 party

We celebrated with a delicious Fedora 29 cake! What are the new features in Fedora 29 were explained, as well as how to create a FAS account and the system badges. I am glad that Fedora Ambassadors from Peru and Poland, as well as Matej from Red Hat Brno met.Food

The organization arranged comfort food more than three times a day, we had extra cookies and water between main menus, plus the cake of course! We tasted delicious Poland food!

Special Thanks

To Filip Kłębczyk not only for being the general organizer but also for giving me one of the best gifts ever! and to Rafał Lużyński for the invitation and the tour around the city.

The uniqueness 

Retro gaming area was a geeky-genius-fantastic-idea 🙂 There was a unique collection of games from the 80’s and we were able to have an entertainment with that treasure.

Advertisements
Posted in FEDORA, GNOME, τεχνολογια :: Technology | Tagged , , , , , , , , , | Leave a comment

Profile of the code on Cirrus

To see the profile on a code, we are going to work with gprof on Cirrus. Let’s use the percolate code: Include the flag -pg for Intel compiler in the Makefile to compile.
After compiling, we will see that gmon.out was created,  as well as its executable:To view the performance data, replace the gmon.out with the executable:

gprof ./percolate

Edit again the makefile with the flags -profile-fuctionts and -O3 as follows:

After recompiling and executing with the Intel profiling, two new files are going to be created *.xml and *.dump. To see the graphic output, run the executable *.sh: A graph will be displayed with the profile of the percolate code:To check the performance of loops, use can use the flag -profile-loops in the Makefile:

-profile-loops=all
-profile-loops=outer
-profile-loops=inner

Each one used separately to produce this outputs:

Screen Shot 2018-11-02 at 11.43.22 AMScreen Shot 2018-11-02 at 11.51.00 AMScreen Shot 2018-11-02 at 11.52.14 AM

Posted in τεχνολογια :: Technology | Tagged , , , , , , | Leave a comment

Unit testing with xUnit practical

This is a lab of the UoE to utilizing xUnit framework to develop unit tests. The experience was inspired by the DNA sequences job. The class of the sequence as follows:

import sys
class Sequence:
    """ DNA sequence consisting of A, C, G, T sequences."""
    WEIGHTS = {'A': 131.2, 'C': 289.2, 'G': 329.2, 'T': 304.2} 
"""dict of str or unicode to float:nucleotide molecular weights"""

    def __init__(self, sequence=""):
     """Initialise Sequence with a string. The string must only
     contain letters in the set a,A,c,C,g,G,t,T.
     :param sequence: sequence
     :type sequence: str or unicode
    :raises AssertionError: if sequence contains an invalid letter
    """
        assert Sequence.is_valid(sequence), \
            "Sequence should only contain A, C, G and T"
        self._nucleotides = sequence.upper()

Python code is in the Python directory.
src/sequence.py contains a Python implementation of Sequence.

To run the tests, and also generate an xUnit XML test report:

Checking the output of the XML test report:

These were the test cases in test_sequence.py:

import unittest
from sequence import Sequence
class SequenceTestCase(unittest.TestCase):
    def test_get_nucleotides(self):
        sequence_str = "GATTACCA"
        sequence = Sequence(sequence_str)
        self.assertEquals(sequence_str, sequence.nucleotides,
                  msg="Nucleotides returned were not those given")

    def test_get_weight(self):
        sequence = Sequence("G")
        self.assertAlmostEqual(Sequence.WEIGHTS['G'],
                            sequence.get_weight(),
                            delta=0.01,
                            msg="Weight returned was unexpected")

    def test_calculate_weight(self):
        sequence = Sequence("G")
        self.assertAlmostEqual(Sequence.WEIGHTS['G'],
                             Sequence.calculate_weight(sequence),
                             delta=0.01,
                             msg="Weight returned was unexpected")                            

(:

Posted in τεχνολογια :: Technology | Tagged , , , , | Leave a comment

FFTW and MelGibson

Printing an image before applying FFTW

Using the FFTW library, we can improve the visualization of the edges of an image. In this case we are going to use Fortran to compile the program fftw1.c using a make file:Now lets run it to have the image indicated in the source codeWorking with Plans to make it better after applying FFTW

Lets create the plan forward and backward

plan_forward = fftw_create_plan(NX,FFTW_FORWARD,FFTW_MEASURE);
plan_backward = fftw_create_plan(NX,FFTW_BACKWARD,FFTW_MEASURE);

Now we are going to perform the plan forward FFT in one dimension

  for(i=0;i<NX;i++)
  {
    for(j=0;j<NY;j++)
    {
      store[i][j].re = data[i][j];
      store[i][j].im = 0.0;
    }
  }
  for (i=0;i<NY;i++)
  {
    for (j=0;j<NX;j++)
    {
      in[j].re = store[i][j].re;
      in[j].im = store[i][j].im;
    }
    fftw_one(plan_forward, in, out);
    for (j=0;j<NX;j++)
    {
      store[i][j].re=out[j].re;
      store[i][j].im=out[j].im;
    }
  }

Let’s perform it in the 2nd dimension

for (j=0;j<NY;j++)
  {
    for (i=0;i<NX;i++)
    {
      in[i].re = store[i][j].re;
      in[i].im = store[i][j].im;
    }
    fftw_one(plan_forward, in, out);
    for (i=0;i<NX;i++)
    {
      store[i][j].re=out[i].re;
      store[i][j].im=out[i].im;
    }
  }

Writing the real and imaginary parts of the image in Fourier space to separate files.

 for (i=0;i<NX;i++)
 {
      for (j=0;j<NX;j++)
      {
      data[i][j] = store[i][j].re;
      }
  }
  pgmwrite("data_real.pgm", data, NX, NY, 1, 2);
  for (i=0;i<NX;i++)
  {
      for (j=0;j<NX;j++)
      {
      data[i][j] = store[i][j].im;
      }
  }
  pgmwrite("data_imag.pgm", data, NX, NY, 1, 2);

Operate on the image in Fourier space; either:

   *  1) cut off high frequencies, or

   *  2) cut off low frequencies

  filter_high_frequencies(store);

  Performing backward FFT in the 1st dimension.

  for (i=0;i<NY;i++)
  {
    for (j=0;j<NX;j++)
    {
      in[j].re = store[i][j].re;
      in[j].im = store[i][j].im;
    }
    fftw_one(plan_backward, in, out);
    for (j=0;j<NX;j++)
    {
      store[i][j].re=out[j].re;
      store[i][j].im=out[j].im;
    }
  }

 Performing backward FFT in the 2nd dimension.

  for (j=0;j<NY;j++)
  {
    for (i=0;i<NX;i++)
    {
      in[i].re = store[i][j].re;
      in[i].im = store[i][j].im;
    }
    fftw_one(plan_backward, in, out);
    for (i=0;i<NX;i++)
    {
      store[i][j].re=out[i].re;
      store[i][j].im=out[i].im;
    }
  }

Storing the final image.

    for (i=0;i<NX;i++)
    {
        for (j=0;j<NX;j++)
        {
            data[i][j]=(store[i][j].re)/(NX*NX);
        }
    }
  pgmwrite("finalFFT.pgm", data, NX, NY, 0, 255);

 Cleaning up the plans:

   fftw_destroy_plan(plan_forward);

   fftw_destroy_plan(plan_backward);

Finally compiling and running and the Mel Gibson image will improve with the functions.

gcc -O2 -L $LIB  -l $PATH-o  fftw1 fftw1.o io.o -lsfftw -lm

./fftw1
Posted in τεχνολογια :: Technology | Tagged , , , | Leave a comment

Calculating PI using MPI

Sequential program

This is the program in C that use the math formula with sums.Parallelizing the code 

  • Parallelizing the code manually:Now we are going to run it and check if all the processors resolve the same answer:
  • Parallelising the code using MPI so the sum can be partially calculated for processes

This time we are going to test with 8 processes

  • Finally, we are going to reduce the partial results into a total calculation of pi considering time of execution

In this case we are receiving the messages indistinctly, but if we need to send it in order, we can set the variable “I” in the receive function.

Posted in τεχνολογια :: Technology | Tagged , , , , , , | Leave a comment

Parallelising Mandelbrot

We have a sequential program of Mandelbrot that take a while when executing, and we can made it in parallel using private and shared variable to check the performance on each one

This is the sequential program C code:and this is the version in C parallelised:Comparing performance

Now lets compare the runtimes between these two options. This is the output for the serialThese are the outputs for parallel ones:


* The parallelization in Fortran as follow:and the output to confirm we did a good job parallelizing: Nested loop

In this case we are going to set two threads that can have other for 3 threads. Then, level 1 is going to handle 2 threads, and at level 2 we are going to have 4 threads!
The answer has to be the same with a lower time:

Posted in τεχνολογια :: Technology | Leave a comment

Starting with OpenMP

We are going to work in DICE and the conventional hello world in C and Fortran:

Hello World (OpenMP with fortran):

The hello world in Fortran 90 as follow:

By default it will take the number of threads as 16:

But if we want to set the numbers to 4, then we can export the desired value 

Hello World (OpenMP with C):

The hello world program as follow with the number of threads set in the environment:

Printing threads inside and outside the critical region in Fortran

Printing threads inside and outside the critical region in C

(:

Posted in Education, τεχνολογια :: Technology | Tagged , , , , | Leave a comment