The design specification and software implementation of a physiological signal-based user-independent emotion recognition system is proposed. The system will have various valuable applications in medicine, computing, education and security domain. Unfortunately, there are many challenges that must be overcome and addressed and this what make the contribution in this field valuable and much needed. The proposed system should operate as a user-independent system, based on physiological signal databases attained from several subjects. Various physiological signals were obtained and measured such as electrocardiogram (ECG), skin temperature, galvanic skin response (GSR), Electromyography (EMG), heart rate (HR), respiration rate (RR), saturation of oxygen in the blood (SPO2), Systolic blood pressure(SBP), and diastolic blood pressure(DBP). To construct the proposed system, the acquired signals went through preprocessing, feature extraction and statistical analysis. A subset of features was selected to be incorporated in the final system due to several technical limitations and design constraint. Using statistical analysis, the system was able to recognize emotion on three basic emotional states namely: anger, joy, and neutral.