Abstract: This paper systematically studied the measurement method of complex permittivity of materials in the millimeter-wave frequency range. Based on the free space-method, a two-tier calibration and the time-domain gating technology were proposed to characterize the dielectric property of materials in the broad frequency range of 75～110 GHz, and an open resonant cavity with a high-quality factor is developed based on the quasi-optical Fabry-Perot resonance principle. An inverse model of the dielectric properties of materials is developed using Gaussian beam theory, which is particularly suitable for the accurate measurement of low-loss dielectric materials. By comparing the measurement results of polyvinyl chloride (PVC) and fused quartz, the consistency of these two methods was verified. In addition, we proposed a de-embedding algorithm for multilayer materials, which can characterize the anisotropic dielectric properties for liquid crystals. The Gaussian beam theory was used to establish an inversion model for extracting the complex permittivity, which was especially suitable for the accurate measurement of low-loss materials.