KEMBAR78
Generic Image Processing With Climb - Slides | PPTX
Generic Image Processing with Climb

           Laurent Senta – Didier Verna
                  April 30, 2012

                        LRDE
                   EPITA Research Lab




                   lrde.epita.fr
Outline
•   Introduction
•   Using Climb
•   Developping Climb
•   Going Further
•   Conclusion




2
INTRODUCTION


3
Generic Image Processing
Images: Roland Levillain. Software Architecture for Generic Image Processing Tools



                                                I       H           B

                                            G                   C
                                                        A
                                                                        D
                                                    F
                                                            E


segmentation


                                                I       H           B

                                            G                   C
                                                        A
                                                                    D
                                                    F
                                                            E

4
Genericity Purpose
Graphic: Laurent Najman. Point de vue d'un théoricien sur l'intérêt de la généricité pour le traitement d'images




                                          algorithms




                     segmentation

                                                                                  S x V x A combinations


                                                                              values
                                                                               type
                        graph               bool grayscale rgb
                2dmatrix
          3dmatrix


                    structures
5
                       type
Climb
• Generic Image Processing library
      – Common Lisp
• Inspired by Olena: C++, 10 years old
      – Dynamic vs Static
      – Usability, maintainability, performance,…
• Still in beta



    Compilers      sbcl
    Dependencies   iterate, lisp-magick, cl-heap, lisp-unit, cl-gtk2



6
Architecture
The different layers of the library




                                            GUI




                                                         Chaining
                                      Image Algorithms
                                                         Operator




                                       Generic types     Morphers




7
USING CLIMB


8
Algorithms



    original     grayscale     Otsu threshold




     jitter    median filter      dilation      erosion


9
Chaining Operator
Chaining algorithms with the $ operator

                (setf   img (image-load ‘images/lena128gray.jpg’))
                (setf   ot-img (otsu img))
                (setf   dil-img (dilation ot-img (4-connectivity)))
                (save   dil-img ‘out/lena-dilated.png’)



                (save (erosion (otsu (image-load ‘images/lena128gray.jpg’))
                               (4-connectivity))
                      ‘out/lena-eroded.png’)




                image-load ‘images/lena128gray.jpg’
                otsu
                dilation (4-connectivity)
                save ‘out/lena-eroded.png’




                ($ (image-load ‘images/lena128gray.jpg’)
                   (otsu)
                   (dilation (4-connectivity))
                   (save ‘out/lena-eroded.png’))




10
Chaining Operator
More complex chaining


                                    erosion



 image-load        to-gray   otsu              diff



                                    dilation




11
DEVELOPPING CLIMB


12
Image Definition
                               I       H           B

                           G                   C
                                       A
                                                   D
                                   F
                                           E

 Image access

      • matrix[x, y] → pixelvalue
      • graph.getNode(label) → nodevalue
      • model[x, y, z] → voxelvalue

      Generalization:   image(site) = value

      Lisp:        (setf (iref image site) value)

13
Values and Sites
 Values:



             bool              grayscale                            RGB

     Site:
                                               I        H       B
                               x
                                           G                C
                                                       A
                                                                D
                                                   F
                                                            E
                y
                    2d-point                           label



               (setf (iref image site)
                     (value-inverse (iref image site)))


14
Browsing Images
         Site-set:
                               x           I       H       B
                                       G               C
                                                   A       D
                                               F       E
     y



           (let ((domain (image-domain image)))
             (loop :for s := (site-set-next domain)
                   :while s
                   …))




15
Browsing Images
           Site-set-window:
                                 x           I       H       B
                                         G               C
                                                     A       D
                                                 F       E
     y



         (let ((neighbors (site-set-window window site)))
           (loop :for s := (site-set-next neighbors)
                 :while s
                 …))




16
Morphers
Transforming images with morphers
              Value morpher         Structure morpher




17
GOING FURTHER


18
Properties
Adapting genericity


                                 algorithms


                                          support:regular

                  segmentation




                                                            values
                                                             type
                     graph         bool grayscale rgb
           2dmatrix
       3dmatrix


               structures
19                type
GUI
Climb based interface




                                                    Value:
                            I       H           B   GTK-BOX

                        G                   C
                                    A               Morpher
                                                D    Resize
                                F
                                        E

                            climb graph

20
Conclusion
Current status


        Image Processing Practitionner                  Algorithm implementor

        • Built-in algorithms                        • High-level domain model
        • Composition tools                               • Generic values
              • Chaining operator                         • Generic structures
              • Morphers                                  • Generic implementations
        • GUI




     Hot topics


                   Genericity            Usability           Performance




21
Thanks for your attention,

     QUESTIONS ?


22
Sources
•    Th. Géraud and R. Levillain. Semantics-driven genericity: A sequel to the static C++ object-oriented
     programming paradigm (SCOOP 2).
•    R. Levillain, Th. Géraud, and L. Najman. Why and how to design a generic and efficient image processing
     framework: The case of the Milena library.
•    N. Otsu. A threshold selection method from gray-level histograms.
•    P. Soille. Morphological Image Analysis: Principles and Applications
•    Roland Levillain. Software Architecture for Generic Image Processing Tools
•    Laurent Najman. Point de vue d'un théoricien sur l'intérêt de la généricité pour le traitement d'images




23

Generic Image Processing With Climb - Slides

  • 1.
    Generic Image Processingwith Climb Laurent Senta – Didier Verna April 30, 2012 LRDE EPITA Research Lab lrde.epita.fr
  • 2.
    Outline • Introduction • Using Climb • Developping Climb • Going Further • Conclusion 2
  • 3.
  • 4.
    Generic Image Processing Images:Roland Levillain. Software Architecture for Generic Image Processing Tools I H B G C A D F E segmentation I H B G C A D F E 4
  • 5.
    Genericity Purpose Graphic: LaurentNajman. Point de vue d'un théoricien sur l'intérêt de la généricité pour le traitement d'images algorithms segmentation S x V x A combinations values type graph bool grayscale rgb 2dmatrix 3dmatrix structures 5 type
  • 6.
    Climb • Generic ImageProcessing library – Common Lisp • Inspired by Olena: C++, 10 years old – Dynamic vs Static – Usability, maintainability, performance,… • Still in beta Compilers sbcl Dependencies iterate, lisp-magick, cl-heap, lisp-unit, cl-gtk2 6
  • 7.
    Architecture The different layersof the library GUI Chaining Image Algorithms Operator Generic types Morphers 7
  • 8.
  • 9.
    Algorithms original grayscale Otsu threshold jitter median filter dilation erosion 9
  • 10.
    Chaining Operator Chaining algorithmswith the $ operator (setf img (image-load ‘images/lena128gray.jpg’)) (setf ot-img (otsu img)) (setf dil-img (dilation ot-img (4-connectivity))) (save dil-img ‘out/lena-dilated.png’) (save (erosion (otsu (image-load ‘images/lena128gray.jpg’)) (4-connectivity)) ‘out/lena-eroded.png’) image-load ‘images/lena128gray.jpg’ otsu dilation (4-connectivity) save ‘out/lena-eroded.png’ ($ (image-load ‘images/lena128gray.jpg’) (otsu) (dilation (4-connectivity)) (save ‘out/lena-eroded.png’)) 10
  • 11.
    Chaining Operator More complexchaining erosion image-load to-gray otsu diff dilation 11
  • 12.
  • 13.
    Image Definition I H B G C A D F E Image access • matrix[x, y] → pixelvalue • graph.getNode(label) → nodevalue • model[x, y, z] → voxelvalue Generalization: image(site) = value Lisp: (setf (iref image site) value) 13
  • 14.
    Values and Sites Values: bool grayscale RGB Site: I H B x G C A D F E y 2d-point label (setf (iref image site) (value-inverse (iref image site))) 14
  • 15.
    Browsing Images Site-set: x I H B G C A D F E y (let ((domain (image-domain image))) (loop :for s := (site-set-next domain) :while s …)) 15
  • 16.
    Browsing Images Site-set-window: x I H B G C A D F E y (let ((neighbors (site-set-window window site))) (loop :for s := (site-set-next neighbors) :while s …)) 16
  • 17.
    Morphers Transforming images withmorphers Value morpher Structure morpher 17
  • 18.
  • 19.
    Properties Adapting genericity algorithms support:regular segmentation values type graph bool grayscale rgb 2dmatrix 3dmatrix structures 19 type
  • 20.
    GUI Climb based interface Value: I H B GTK-BOX G C A Morpher D Resize F E climb graph 20
  • 21.
    Conclusion Current status Image Processing Practitionner Algorithm implementor • Built-in algorithms • High-level domain model • Composition tools • Generic values • Chaining operator • Generic structures • Morphers • Generic implementations • GUI Hot topics Genericity Usability Performance 21
  • 22.
    Thanks for yourattention, QUESTIONS ? 22
  • 23.
    Sources • Th. Géraud and R. Levillain. Semantics-driven genericity: A sequel to the static C++ object-oriented programming paradigm (SCOOP 2). • R. Levillain, Th. Géraud, and L. Najman. Why and how to design a generic and efficient image processing framework: The case of the Milena library. • N. Otsu. A threshold selection method from gray-level histograms. • P. Soille. Morphological Image Analysis: Principles and Applications • Roland Levillain. Software Architecture for Generic Image Processing Tools • Laurent Najman. Point de vue d'un théoricien sur l'intérêt de la généricité pour le traitement d'images 23

Editor's Notes

  • #3 Starts bydescribing Image Processing and whywe are looking for genericityThenwe’lltake a look atClimbfromdifferent point of view: First: as a IP researchersthat uses the toolsweprovide to transformits images Second: as a Developperthatwants to implements new algorithms in the libraryFinally, I’ll show you prototypes that are not fully part of the libraryyetDon’thesitate to ask question any time if I’m not clear
  • #4 Let’s talk about Image Processing
  • #11 Usingsetf and variable to store the intermediateresultsChaining all the algo call in one bigprocedureHowever, the first method: repetitive, many variable, namemistakes = errorsthatcouldbe hard to find the second methodmay push the algorithmparametersaway, hard to understand and modificateWhatwewant to do as IP researchers:chainmanyalgorithm andthe librarywouldimplicitlyunderstandthateachresultismeant to bepassed to the nextprocessingThanks to the Lisp macro system, Climbprovides the $ operatorthatwill replace the first parameter of each call by the result of the previous one.Usingthis IP-orientedsyntax, wecanwritereadblechain of algorithmwithouthaving to deal withmanyrepetition > Best of both world[Demo: erosion](loader le lena-otsu!)
  • #12 Now, sequence of algo: too simpleOftenyouwan to branch the processing in order to construct more complexoperatorsHereis an example of a basic border detectionfilterthatworks by computing the diffbetween the eroded and dilated version of the same image.$-operatordedicated to IP, soanothersyntaxisadded to support thisparadigmwith the 2 slash operator[Demo]I’mgoing to applythisfilter on a a new image,Copy and paste the code fromels.rtWhathappen ?Add the //But I’mmaywant to call functionthat are not part of the image processlikeloggingAdd the ‘(timer-start) ‘(timer-print ‘’Time:’)CallDetailresultsShow the printed messageThis toolis important to us because shows capabilitiesof Lisp to transformitselfconstructDSL not simplydedicated to complex programmer trick but canbeusedto adapt the language to the userswe’reaiming for.
  • #13 Well,nowlet’s a look at the Genericity layer and how wecanadd new algorithm in the library
  • #16  Site-set, set of sites used to browse an areaThey are iterator that returns a site each time you call site-set-next[Slide1-2-3]retrieve all the site for a given image[Slides]Explain line image-domain, call code on s (could be the previous code) no-info underlying typeDemo: InvertNow that we can browse a while image, we also want to browse a specific area,That we call the neighborhood of a site.
  • #17 Anothercommon patternBrowsing a neighborhood allows us to compute local information,And for example, in the segmentation algorithm, find the sitesThat are close, in the value space but also in the structure spaceBrowse a neighborhood: site-set centered on a site and retrieve all it's neighborhoodsUsed for example to compute the mean value in the mean filter or in the erosion/dilationTo spread the values to their neighborsExplain lineAgain, no infoLet’swrite a bluralgorithm[DEMO]
  • #18 In order to finish with the climbdeveloppement, let’stake a look atmorphers.Morpher are objectsbuiltaround an image thatwilltransformsome of it’spropertiesdynamically.Values: transforme the valuesFor example: gray morpher, convert values, image saw by the outside world as grayscaleStructure: Transform the structuresFor example: restrictmorpher, the image saw by the outside world as a smaller oneAvoidWaste of memorySimplifygenericitysinceyoucaneasilyconvert an image that do not fit the prerequisite for an algorithmA thresholdalgorithmthatneeds a grayscale imageAn algorithmthatneeds an image with a size being a power of 2Slower ? Not necessarily: morpherblurwithmorpherresize[Demomorpherblur]
  • #19 NowI’mgoing to talk about prototype thingswe’reworking on
  • #21 these algo are available in a GUI Provide the algorithm without knownledge in programming interesting: based on the generic types defined in ClimbClimb based GUI: climb graph for the algo call and the gtk displaymorpher used for image resizing from the computation to the display (no duplication)Interesting: library generic enough to allows unusual uses of the types[Demo] run the previous $-chain using the GUIAdvantage over $: Cycle for density computation,…
  • #22 Conclusion:Even if Climbis not yet a stable generic IP library, we have been able to developtoolsthat shows how Lisp canbe an interestinglanguage in a domainwere C and C++ are king.Westill have somedevbeforebeing able to release it, howeverheresome point of enhancementwe’dlike to be able to work ongenericity and usability: automorphing performances: bench the wholelibraryusability: goingfurtherwith the gui by discoveringautomatically the algorithms and parameters
  • #23 Label, applicable ?, rIsize