Aim: All physicians must care for some patients who are perceived as difficult because of behavioral or emotional aspects that affect their care. Reasons may be delivered from patient, physician, or health care system. To identify reasons behind perceiving some patients as “difficult” from a family doctor’s point of view; propose solutions for dealing with, and ultimately accepting and understanding them. Method: Cross sectional descriptive study. Participants: Data was collected from seventy board-certified family doctors currently working in primary care services. A structured questionnaire with answers scored on a 2 point scale (agree/disagree) was used to assess the perception of difficult patients. Twenty questions were related to patients, 8 were related to doctors, and 8 were related to administrative system. Results: The majority of doctors agreed that patient characteristics that render them difficult included psychological disorders (95.7%), life stresses (95.7%), social isolation (87.1%), multiple physical problems (78.6%), chronic diseases (82.9%), inability to communicate own needs (80.0%) and unrealistic expectations of the patient (77.1%). Most participants considered greater work loads (81.4%), lack of job satisfaction (72.9%), psychic condition of the doctor (68.6%), lack of training in counseling (78.6%) and communication skills (74.3%) were the main physicians reasons behind perceiving patients as difficult. Factors related to the administrative system were mainly absence of strategies to deal with difficult patients (91.4%), lack of a legal policy in the clinic (82.9%), free health services (82.9%), improper appointment system (81.4%), failing of registration to a single doctor (80.0%), unavailability of social services (72.9%), and untrained receptionists (71.4%). Conclusion: Our study demonstrated that the “problems” do not lie exclusively with the patients. Doctors should analyze difficult patient encounters to identify the causes behind them. In the process they will possibly overcome their negative feelings and discover some of their educational needs