Background: The need for Home Based Primary Care (HBPC) is increasing due to rapid population aging and the increased number of elderly people with mobility problems. This study aimed to confirm the need for HBPC and suggest a family medicine residency training program to enable HBPC expansion. Methods: Individual in-depth interviews were conducted with the physicians who practiced HBPC at least once a month. Two focus group interviews were conducted of the Family Medicine residents with or without HBPC experience. All conversations were recorded and transcribed using automatic recording application after the participants provided informed consent. The data were analyzed using qualitative analysis methods. Results: All interviewees agreed about the need for HBPC to strengthen the doctor–patient relationship and the role of the primary care physician. Residents with HBPC experience had a positive perception of it. For training, a dedicated residency program director was required. Furthermore, standardized guidelines and protection against possible legal issues during the visit were required. Additional discussions are needed regarding the appropriate time for training and the scope of work as a primary care physician. Conclusion: To strengthen HBPC and nurture human resources in a super-aged society, a HBPC training program is necessary. A high-quality HBPC training environment requires a dedicated residency program director, standardized guidelines and educational programs developed through cooperation with a primary care clinic, and an improved payment system.