As input I have a set of triangles that form concave mesh with holes. Each triangle consist of three vertices: (vi,vj,vk),… No adjacency information is provided. Algorithm have to “union” the set of triangles with the same normal into polygon. So output is (pi,pj,pk,…,ps),…
For example(see picture below), let say we have mesh that consist of triangles
(v0,v1,v4), (v1,v3,v4), (v1,v2,v3)
(v2,v6,v4), (v6,v5,v4).
As output we have:
(p0,p1,p4)
(p1,p2,p3,p4)

I’m looking for efficient algorithm that solves problem described above. Any suggestion, tips or articles is appreciated.
Look at adaptive mesh coarsening algorithms. These are typically used in advanced software for Computational Fluid Dynamics (Ansys CFX, CD-Adapco Star CCM+ etc.) / Structural Dynamics (Anasys et al.). Where automatic refinement and coarsening of a given mesh is advantageous.
Some free papers to look at on this subject, that will give you a strong starting point are:
https://cfwebprod.sandia.gov/cfdocs/CCIM/docs/coarsening.pdf
http://tetra.mech.ubc.ca/ANSLab/publications/coarsen.pdf
http://www.cs.cmu.edu/~glmiller/Publications/MiTaTe98.pdf (This is fairly mathematical)
Additional Google searches in the area of adaptive mesh refinement algorithms will reveal similar papers on the subject.
Edit. A basic and well established method for the adaptive mesh coarsing is the Edge Collapse Method which involves reducing an edge to a single vertex and thereby removing two elements. Li X., Shephard M.S., Beall M.W. “3-D anisotropic mesh adaptation by mesh modifications.” Computer Methods in Applied Mechanics and Engineering, 2004. has a great coarsening algorithm which is defined in pseudo-code as
This is taken from an impressive Masters thesis I have used in the past. It can be found here
I hope this helps.