The present courthouse, where the county government is located, is an adaptation of the old Hyatt Hotel built around 1890. It faces the "round-about" in the center of Ellijay. Through the years citizens have maintained an active interest in their government and a strong two-party system exists.

The county government is under the leadership of an elected three-member Board of Commissioners, with the elected chair serving as County Manager and an appointed Clerk to run the day-to-day business. Numerous other county officials are elected, including Tax Commissioner, Clerk of Court, Probate and Magistrate Judges, and Sheriff. The City of Ellijay is governed by a Mayor-Council (five members) form of government elected by the citizens of the city. East Ellijay has an elected mayor and four-member council. Several other city officials are appointed. Local law enforcement is handled in the county by an elected sheriff and his deputies, while each city has a police department directed by a chief appointed by the mayor and council. An Emergency Management Agency handles fire, rescue, and ambulance services for the city of East Ellijay and county with a county-wide 911 enhanced service.  The city of Ellijay also has fire and ambulance service.

Fannin, Pickens, and Gilmer Counties form the Appalachian Judicial Circuit. At present, there are two judges presiding in the Superior Court of Gilmer County. The office of Clerk of Court is a part of the judicial system. The Probation Office and Pre-Trial Services Offices are adjuncts of the Superior Court. A District Attorney and several Assistant District Attorneys serve the Appalachian Circuit. The county has a Probate Court, a Juvenile Court (the Circuit Juvenile Judge presides) and a Magistrate Court. Each city has its own appointed municipal judges.