Question

How to make the first matrix look like the latter?

The code of the first matrix is:

$$F(x)=\left(
\begin{array}{cc}
\frac{x}{2} & 0 \\
0 & x^2\\
\end{array}\right)$$

Answer 1

The second (wanted) example uses a different math font family. These define the thin brackets. I’m not a font expert, so I can’t tell you which font this example uses.
I can tell you, I know the Math-Fourier font also uses thin brackets. This is what I’ve used to recreate your second example:

\documentclass[english]{article}
\usepackage{fourier} % The Math font
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage{babel}

\begin{document}
\[
F(x)={\scriptstyle \left(\begin{array}{cc}
{\textstyle \frac{x}{2}} & {\displaystyle 0}\\
\vphantom{{\displaystyle W}} % uses some vertical spacing
                             % between the first and second row
{\displaystyle 0} & {\displaystyle x^{2}}\end{array}\right)}\]

\end{document}