This is a good question.

An answer to this question is given in

P. Gatto, J.S. Hesthaven, Numerical approximation of the fractional Laplacian …, 2000:

http://infoscience.epfl.ch/record/199424/files/FracLap.pdf

See p. 6 for Green's function for the heat operator (equivalent to applying a time-varying Gaussian filter to a noisy image).    See p. 7 for details.

See, also,

Q.-Y. Guan, Z.-M. Ma, Reflected symmetric $\alpha$-stable processes and regional fractional Laplacian:

http://www.amt.ac.cn/member/mazhiming/papers/reflected.pdf

Thanks! I read them, but there is no good description on the Green's function.

The first paper used extension argument and it's operator is given by the eigenvalues and eigenfunctions.

The second one is good.  but is there is still no description on the 

Green's function, especially near the boundary.

Should be there some thing not to be seen easily?

Best

Huyuan

Hello,

the paper by P. Gatto and J.S. Hesthaven uses the definition of the fractional Laplace operator based on the eigenvalues and eigenfunctions of the standard Laplace operator. This definition is not equivalent to the "regional" definition; see

http://link.springer.com/article/10.1007%2Fs10208-014-9208-x

for references. The latter in turns motivates the numerical scheme explored by P. Gatto and J.S. Hesthaven.

I have the impression that the following reference by L. Cafarrelli and P. R. Stinga could be useful:

http://www.sciencedirect.com/science/article/pii/S0294144915000153#

Please tell me if this helps: [email protected].

Enrique.

Look up the papers of:

Bogdan

Kulczycki

Serra

Ros-Oton

Thanks a lot!