Abstract
Recent years have witnessed a surge in telerehabilitation and remote healthcare systems blessed by the emerging low-cost wearable devices to monitor biological and biokinematic aspects of human beings. Although such telerehabilitation systems utilise cloud computing features and provide automatic biofeedback and performance evaluation, there are demands for overall optimisation to enable these systems to operate with low battery consumption and low computational power and even with weak or no network connections. This paper proposes a novel multilevel data encoding scheme satisfying these requirements in mobile cloud computing applications, particularly in the field of telerehabilitation. We introduce architecture for telerehabilitation platform utilising the proposed encoding scheme integrated with various types of sensors. The platform is usable not only for patients to experience telerehabilitation services but also for therapists to acquire essential support from analysis oriented decision support system (AODSS) for more thorough analysis and making further decisions on treatment.
1. Introduction
Resource deficiencies in the healthcare industry to cater the rapid changes in demography entail the development of new rehabilitative practices [1] enabling therapists to provide their services to remote communities. Since the first article using the term “telerehabilitation” published in 1998 [2], this new rehabilitation practice paradigm has been widely studied for its capabilities in reducing patients’ commute to hospital and saving time and cost for both clinicians and patients [3]. Telerehabilitation can be defined as a set of instruments and protocols aimed at providing access to rehabilitation services for patients at geographically distant locations [4].
Carignan and Krebs [5] mentioned that one of the challenges in developing a telerehabilitation system is to reduce the delay in the data communication and processing, especially where real-time biofeedback is required. As is observed, the major delay in the telerehabilitation is caused by the low bandwidth of the Internet, large amount of data, and the low computational power of the computer used to process captured data. Therefore, the delay resulted from the increase in users can be partially addressed by increasing the computational power of the computer hosting the server side of the telerehabilitation system. Furthermore, the increase in users is associated with a significant increase in captured data from various sensors, which requires the flexibility in capacity of the database. Additionally, applying pervasive computing in healthcare, such as telerehabilitation, is becoming popular in the recent years [6, 7], which leads to a challenge on how to integrate existing technologies to a telerehabilitation system.
With recent developments in ICT, mobile cloud computing (MCC) [8] has emerged to bring cloud computing [1] power to mobile devices. MCC [8] can be considered the ideal candidate to accommodate majority of the above criteria.
Firstly MCC inherently provides architecture to support various mobile devices, such as smartphones, tablets, and laptops, which means that users of telerehabilitation services can readily access the service with less restrictions on the time and location.
Secondly, similar to generic cloud computing, MCC also provides an elastic approach to support the increasing number of users. One example is Amazon Web Services (AWS) [9], which enables users to expand the computation power and data storage space easily. More importantly, users deploying their services in cloud only pay for the resources they use, which is also called “pay-as-you-go” (PAYG), thereby reducing the cost significantly compared to traditional approach to deploy services.
Thirdly, the computation power in MCC enables performing calculation fast enough to offer patients real-time biofeedback, which is critical in telerehabilitation [10] to improve the effectiveness of telerehabilitation session [11–13]. Because of the limited computational power and battery resources, mobile devices are unlikely to handle this task for a long time. Therefore, offloading the computation tasks to the cloud reduces the load in mobile devices and enables telerehabilitation users to access the service as long as possible.
Although MCC has provided a well-designed platform to establish telerehabilitation services, there exist some gaps to be solved for this particular application. One of the most critical issues is how to reduce the energy consumption in mobile devices so that patients are able to access telerehabilitation services as long as possible. In addition, another open question is how to design a telerehabilitation platform that caters for various types of sensors to collect sufficient data in a short period so that automatic assessment and analysis of patients’ performance can be performed in the therapeutic procedures as an ongoing treatment scheme.
This paper primarily seeks the answers for the previously mentioned questions. The main contribution of the paper is a multilevel data encoding scheme for optimisation of computational offloading and data transferring. Although a vast array of studies discussed about strategies to optimise the computational offloading and data transferring, we found no study focusing and customising the algorithm for biomedical signals. In order to answer the second question, we proposed architecture of MCC-based telerehabilitation platform to support various types of sensors. The analysis oriented decision support system (AODSS) and the security service layer (SSL) are also integrated in the platform. The former utilises the huge amount of data to analyse the performance of patients from different levels of granularities, while the latter protects the information of data spanning from acquisition, storing, and transferring throughout the platform. These two are research challenges in conventional clinical decision support system (CDSS) [14] and will be discussed in Sections 4.5 and 4.6, respectively.
The rest of the paper is organized as follow. Section 2 lists related work, followed by our main contribution, the encoding scheme, and approach to optimise the energy of computation and data handling, in Section 3. Further more, an overview of the system architecture, as well as the concepts of AODSS and SSL, is presented in Section 4. Additionally, the simulation and real-data experiments regarding the multilevel encoding scheme are presented in Section 5 where the demonstration of the interfaces of the telerehabilitation platform is included. The final section presents a discussion on the overall approach and concluding remarks.
2. Related Work
A number of studies [2] have been conducted in the past few decades on developing efficient telerehabilitation systems. Initially, most of the telerehabilitation was based on the plain old telephone systems, also known as POTS [15]. As the technology advanced, more sophisticated methods were introduced into the telerehabilitation process. For instance, in 2002, the utilization of interactive video was proposed by Clark et al. [16] in order to provide teletherapy specifically for stroke patients where two videophones were set-up both in the therapist’s office and at the patient’s home, enabling the communication between both parties for that particular physical rehabilitation. Using this method, the patient was able to perform the walking activities as well as take care of themselves at home independently. In another reported work by Liu and Miyazaki [17], a video-conference device based rehabilitation system was deployed between the staff in the university of Albert and the clinicians/students in rural areas. The deployed system was aimed at being utilized in the education field supervising the students and the clinicians rather than in the application of treatments where 96% of telehealth sessions were successfully reported. Russell et al. [18] have developed a telerehabilitation software based application to record the video from the video conferencing link in order to remotely monitor the patients with gait conditions. They have evaluated the proposed system in terms of the accuracy and the reliability where the outcome of the gait assessments results through the internet was very close to the traditional method with less than 1 point on Gait Assessment Rating Scale (GARS). More examples can be found in [19].
Due to recent developments in cloud and mobile cloud computing technologies, more applications have been developed in the telerehabilitation and e-Health arena. Popescu et al. [20] has integrated a PC, a Polhemus tracker, and a haptic control interface together with force feedback in order to create a platform for the telerehabilitation service between the patients and the therapist through remote monitoring. Other than that, a web based telerehabilitation system was developed by Reinkensmeyer et al. [21] to enable the poststroke patients to perform rehabilitation exercises while being monitored by the therapist through the therapy games activity where a force-feedback joystick was used to provide resistance. Additionally, a virtual environment with a motion capture device (approximately $8500) was designed by Holden et al. [22] to offer a telerehabilitation environment for patients at home and, at the same time, the therapists were able to monitor patients’ performance and hold video-conference remotely. The improvement of assessment scores during clinical trials illustrated the effectiveness of this system. Moreover, Sultan [23] discussed about opportunities and challenges in cloud-based healthcare provisioning and gave an example of a successful model: the pilot e-Health system at Chelsea and Westminster hospital. Wu et al. [24] proposed two approaches to perform offloading on multiple servers for mobile healthcare systems. The first approach is optimized for stabilization and the second approach is optimized for energy efficiency. Liu and Park [25] specified characteristics of e-Health applications and proposed an adaptive architecture design for e-Health Cloud, Computing and Networking Solutions. Hendrick et al. [26] proposed a platform that is able to support medical records of an entire country. He et al. [27] proposed a private cloud platform architecture which includes six layers utilizing technologies such as MQ, load balance, plug-in algorithm, and cloud-storage. Wu and Khoury [28] designed a web service broker that can facilitate the data exchange between healthcare providers in order to achieve a complete patient medical records prior to treatment.
More relevant to our contribution, a number of approaches have been raised to estimate the energy consumption for mobile devices in mobile cloud computing applications. Wolski et al. [29] introduced a framework for offloading decision making by taking the bandwidth between the remote and local computer into account. Furthermore, Karthik [30] proposed a method to estimate the energy consumption by considering the computational power and bandwidth. More examples can be seen in [31].
3. Multilevel Data Encoding Scheme
This section is dedicated for solving one of the limitations, namely, limited power for computing and data transferring, in applying mobile cloud computing in telerehabilitation field by utilising the characteristics of biomedical data collected from various types of sensors.
3.1. Protocol
Due to the limited battery resources in a mobile device, how to reduce the power consumption is an open question in MCC field. In this paper, we propose a multilevel data encoding scheme so that the data can be encoded differently to reduce the quantity and time during transfer, thereby reducing the power utilized to transfer data. However, different encoding schemes require various computational times. Generally speaking, if the amount of data after encoding is smaller, the time utilised to encode the original data is longer. Therefore, it is crucial to find an approach to determine which level of the encoding scheme should be utilised.
Figure 1 shows the data encoding schemes for two types of telerehabilitation exercises, including motion rehabilitation (left side of the triangle) and respiratory rehabilitation (right side of the triangle). The encoding scheme in the higher position indicates that the data amount after encoding is smaller than approaches in lower levels. For instance, motion trajectories involve three-dimension data in a period of time and elbow points encoding scheme reduces the number of samples in motion trajectories, thereby reducing the amount of data. Furthermore, shape model reduces the dimension from three to two and performance measurement enables a telerehabilitation exercise to be represented by a single number that is meaningful to clinicians. Similar concept can be applied to respiratory data captured from Doppler radar, where the raw data (two-dimension) for respiratory exercises is radar signal. After applying arctangent demodulation, the exercises can be encoded by a single dimension data. Eventually, a performance measurement score can also be derived.
For instance, in human motion capturing, the details of each encoding scheme are introduced as follows.
(i) 3D Motion Trajectory. In the majority of the optical-based portable motion capture devices such as Kinect and Creative Senz3D, human motions are captured in terms of the positions of the joints in the forms of , where , , and are 3D positions of th joint () on the , , and axes at time in the traditional Cartesian coordinate system. Compared to VGA video, a frame of 3D trajectories for joints is bytes, where, for example, in Kinect version 1 and in Creative Senz3D. Since both videos and trajectories are collected from motion devices, we assume that the same power or energy is required to retrieve the data.
(ii) Elbow Point Technique. In a motion trajectory, each point has its own importance and its contribution to the shape of trajectory is different from others. An array of points lying on a straight line can be represented by only two points at the two end points of the line. Therefore, points at the middle of the straight line can be removed to reduce the computational cost and data storage. Points lying on a curve are called “elbow points” which are essential points to form the shape of the trajectory [32]. Discrimination of “straight points” and “elbow points” can be based on curvature. Curvature at a point is defined as where A point is marked as an “elbow point” when its curvature is larger than a specific threshold . Conversely, a point is marked as “straight point” if its curvature is larger than or equal to zero and less than the specific threshold . In elbow method, points with curvature will be removed from the trajectory. The original trajectory can be approximately reconstructed from the new trajectory if the coordinates of the elbow points and their sequential orders are determined. The new trajectory which includes only elbow points obviously requires less computational cost than the original trajectory. We illustrate this technique in Figure 2 for the 2D case. The technique can be applied in 3D case [32]. Here, we plot the function where and . As the figure shows, the new trajectory formed by elbow points is almost identical to the original trajectory with all the points. In this example, we use the value of 0.05 for the threshold , and hence half of the points from the trajectory have been removed. Depending on the application, this threshold value can be chosen accordingly.
(iii) Shape Model. To tackle some situations in which the bandwidth is insufficient to transfer elbow points, shape models can be further derived to encode motion trajectories. Apart from the curvature in (1), torsion is needed since the motion trajectories are in three dimensions. It is derived aswhereis the jerk of the motion trajectory.
From the derivation, it is observed that the motion trajectory is encoded from three dimensions (, , and ) to two dimensions ( and ).
(iv) Performance Measurement. Two methods can be utilised to encode and measure the performance of the telerehabilitation exercises, including smoothness based and elbow point based measurements. As for the former [33], it is specially designed for telerehabilitation service users with neurological movement disorders, such as dyskinesia, which involve large amplitude involuntary movements, which leads to less smooth motion trajectories than healthy people. Due to the fact that the shape model [34] is very sensitive to noise in motion trajectories, the submovements and jerky movements are also shown in the shape model. As a result, the entropy of the shape models of these trajectories are computed to represent the severity of involuntary movements, thereby indicating the ability of patients to perform telerehabilitation exercises given by their therapists. Another method can be used for this encoding scheme which is the algorithm in [32]. In this approach, the authors used longest common subsequence (LCSS) to match the trajectory from the patient and the corresponding one from the therapist (model motion trajectory) and gave a score for performance measurement.
Taking remote monitoring of human respiratory function using Doppler radar is another example to demonstrate the multilevel data encoding scheme. The methods used to perform encoding in various levels are shown as follow.
(i) Raw Data of In-Phase and Quadrature-Phase Signal. From a direct-conversion architecture of continuous wave Doppler radar, the motion from the abdomen due to inhalation and exhalation activities during breathing causes a shifting in the transmitted wave in terms of phase. Thus, the reflected signal received at the receiving ends will be a phase modulated signal that consists of the information of the motion during breathing. The received signal is denoted as where is the phase noise of the signal source, a nominal distance of , and the time varying displacement of due to corresponding respiration activities. To overcome the null problem, a quadrature receiver is used where the signal will be further represented with the and signals denoted as
(ii) Arctangent Demodulation. Depending on the aim of the application, and signals can be combined using arctangent demodulation for a better accuracy and system requirements (system with bandwidth constraint). The combination of both signals can be represented as
3.2. Determine Encoding Level
The introduction of various encoding approaches naturally raises the question of determining the encoding level that should be used depending on the computational power and the speed used to upload data to the cloud from the mobile device. The work in [30] introduces an approach for offloading decision by estimating how much power is used for computations and transfer of data. We adopted this method to determine which encoding level should be utilised with respect to the bandwidth. Notations utilised in the estimation process are shown in Table 1. Unlike the approach introduced in [30], the computation time in the server is not considered in our case. The reason is that the data communication between the server and the mobile device uses asynchronous channels which means the channels are not blocked when the server is engaged in the computation. Therefore, there is no idle time in the mobile device. Further in our case, it is hard to estimate the number of instructions required by the computation for data encoding. As a result, we directly record the numbers of unit energy consumption () utilized to perform encoding for each frame of data, which can be retrieved automatically by the system.
The formula to calculate data size is
The formula used to calculate local computational power with scheme is where ; is the average power consumption to determine whether a point is an elbow point and it includes the calculation of curvature; is the average power consumption to calculate torsion value of 1 point; is the average power consumption to calculate the performance.
The formula used to calculate data transfer power is where is the energy consumption for uploading data in 1 second and is the network speed. Table 2 summarises the power consumption of various encoding schemes and encoded data transfer for motion telerehabilitation.
The following cost function is used to determine which encoding scheme is to be used:
In some cases, if and , we select so that more data can be transferred to the cloud by consuming similar power. Here is a small constant value.
4. System Architecture
The severity of certain medical conditions can be evaluated from various aspects. For example, respiratory disorders [35] (i.e., dysfunctional respiratory and chronic obstructive pulmonary disease) can be assessed through breathing and motion monitoring and, in a contrasting condition, both hand and limb are affected by stroke. Therefore, in the first example, we may use chest straps or a Doppler radar to detect the respiratory function with inertial measurement unit (IMU) (BioKin in our case [36]) to monitor limb movements. In the second example, Kinect and Creative Senz3D camera can be utilised to monitor the movement of limbs and hands for assessment purposes. These examples indicate that it is essential to have a telerehabilitation platform catering for various sensors aiming at capturing various physiological measures. In this section, we propose architecture for a MCC-based telerehabilitation platform, as well as the implementation details of the prototype system for testing purposes. Although we utilise Kinect, Creative Senz3D camera, Doppler radar, and BioKin as sensors in the prototype system, sensors such as EMG and ECG can also be supported with minimal modification.
4.1. System Platform
As for the platform to deploy the server side of the proposed system, we selected Microsoft Windows 2012R2 for its stability, ease to configure, and being well supported by various cloud computing providers, such as Amazon EC2. In addition, the programming language utilised to develop the overall system was C# due to the following three reasons:(i)The sensors utilised in our telerehabilitation system support C#, especially the Microsoft Kinect, which has SDK (v2.0) for C#.(ii)C# is well supported by Windows operating systems.(iii)The Windows Communication Foundation (WCF) can be integrated as the basic communication framework for the overall system, which can be easily developed by using C#.
Therefore, as per the selection of deploying platform and programming language, the integrated development environment (IDE) utilised to implement the whole system is Microsoft Visual Studio 2013. In order to use the latest functions of C#, we adopted .Net 4.5 as our common language run-time (CLR).
4.2. Web Service Style
To provide web services, two principles in [37] (or called tools to design web service in [38], architectural style [39], and so on) were extensively utilised, including Simple Object Access protocol (SOAP) [40] and Representational State Transfer (REST) [41]. Although both are able to be the architecture for data communication in web service, the latter is more suitable in our system for its capability of reducing the delay. According to [37], the message size and processing and transmission time of REST web service was 9 to 10 and 5 to 6 times, respectively, shorter than that of SOAP based ones, since REST protocol does not need to format payload data as formal as SOAP, thereby reducing the overhead and processing time to parse XML. Eventually, the delay of the whole system can be minimized from a data communication perspective.
Due to the selection of REST, which is HTTP-based, we selected WSDualHttpBinding as binding method for contracts in WCF for its ability to provide asynchronous communication by simulating duplex channels to separate the requests and responses. Therefore, even with some delay in the cloud, the channel is less likely to be blocked compared to single channel model.
4.3. System (Hardware) Overview
From Figure 3, we can see that there are three main components in the telerehabilitation platform: therapist side, server side, and patient side. Except for the server side, there is only one instance of the other two components in the figure, which can actually be multiple instances simultaneously, representing a number of therapists and patients accessing the telerehabilitation services at the same time.
At current stage, due to the limitation of SDKs and drivers of sensors, especially Kinect, Creative Senz3D, and Doppler radar, they are unable to connect to tablet and mobile phone. However, it is noteworthy that a trend has been seen, in recent years, to integrate these sensors into mobile devices. For example, Dell Venue 8 7000 series tablets have integrated Intel RealSens technology to support depth cameras which is conceptually similar to the concept of Kinect. HTC M7 smartphone integrated two cameras to provide depth images which also is a similar concept to the LeapMotion or similar in function to Kinect. Therefore, our proposed telerehabilitation platform not only caters to the existing sensors, but also can be utilised when the sensors are integrated into mobile devices.
Four types of sensors are shown in this paper as examples. Firstly, Microsoft Kinect version 2 is one of the sensors used in our telerehabilitation system. The main function of this device is tracking the 3D positions of 25 built-in joints or multiple self-defined markers with the sampling rate of 30 Hz. The details of the specification of the second version Kinect can be retrieved in [42]. Secondly, BioKin [43, 44] is aimed at capturing human movements with wearable IMU sensors to facilitate exercises typically for remote telerehabilitation. As an ambulatory system possibly used in nonclinical settings, BioKin can provide complementary services to communities with limited access to infrastructure such as gait laboratories. It consists of an inertial sensor, self-contained triaxial magnetometer providing 150 Hz sampling frequency. Thirdly, the Creative Senz3D is an optical sensor which is capable of acquiring depth images and specialised for human body tracking and gesture tracking. While Kinect is designed for full-body capture, the Creative Senz3D is optimised for short-range gesture interaction and, consequentially, it is perceived as a better alternative to hand measurements. Depth images of the Creative Senz3D have the resolution of 640 × 480 pixels and are refreshed with the frequency of 30 Hz. Lastly, a 2.4 GHz quadrature based Doppler radar is a noncontact measurement device that is capable in acquiring the human physiological sign such as respiration and heartbeat motion. Measurements of the signals can then be derived into different forms of information for further analysis. System measurement consists of the time series of in-phase () and the quadrature-phase () signals which is then sampled at 1000 Hz. The sampling rate of the system can be reduced to as low as 30 Hz depending on the aims of the application. The obtained measurements can be further used for but not limited to information extraction [35, 45] and refining diagnosis procedure and as external input to trigger the biofeedback mechanism such as in sleep apnoea and sudden infant death syndrome (SIDS) detection applications.
These sensors can be used by the therapists to record model motions and respiratory data as examples or references for patients to follow and compare, while collecting data for exercise performance assessment and further analysis in patient side.
4.4. System (Software) Architecture Overview
In this subsection, a detailed software architecture of the system is provided as follows.
(i) Therapist Side. Primary focus of this side is on the inclusion of patient profile management, building exercise models (example can be found in Figure 10), and visual (online or off-line) review of the exercise data and analysis result collected from various sensors and AODSS (refer to Section 4.5). Exercise models built in this side have two major aims. First of all, the models can be downloaded by patients and utilised as a guidance in performing telerehabilitation exercises. Secondly, models built by therapists can be used as references for telerehabilitation exercise performance evaluation. As for the data flow, except for life streaming, it is introduced in the patient side.
Figure 4 shows the data flow between the cloud (server side) and the therapist side. Symbols in the graph with different colours indicate the data and requests from various sources and their corresponding responses. Here squares and downward triangles are the encoded version of data represented by circles, while upward triangles and diamonds represent database operations without storing new data. Additionally, as for the responses, there are two methods to display information. One is showing a confirmation statement to notify the therapists whether the requested operation is successful or not. The second approach is visually displaying information obtained from the cloud, such as histograms. In addition, the data flow of off-line review (with green symbols) in Figure 4 is an abstract process and detailed information is discussed in Section 4.5.
(ii) Patient Side. The patient side of our platform provides interactive telerehabilitation services and passive exercise monitoring capabilities. The data flow is shown in Figure 5. For interactive telerehabilitation, first of all, the patient requires the exercise model created by a therapist from the cloud (shown with blue symbols), which is later utilised in the rehabilitation exercises as references and streamed to the patient’s mobile devices (with purple symbols). After that, the performed exercise motions of the patient are recorded and sent to the cloud (with red symbols). It is noteworthy that, instead of sending video or audio data like typical telerehabilitation systems, the encoded data is sent in our system.
When the cloud (server side) receives the recoded exercise information, signal processing techniques are applied to filter out the noise and also to extract relevant features. The data is stored in patient exercise database for off-line review and also used to provide corresponding biofeedback particularly for performance measurement and assessment through the comparison between patients’ acts and therapists’ models. The biofeedback is denoted with blue triangles and presented in various forms (refer to Section 4.7). Lastly, if therapists have enough bandwidth and choose to review patients exercise online, they just have to register a channel, through which the processed motion information is forwarded by the cloud to the therapists. As for the passive exercise monitoring, the data follow the path indicated by red symbols.
In addition, to solve the disconnection issue in mobile terminals, a small local database is maintained in the patient side to store the most recently used model motion information and patients’ exercise motion data. Therefore, even when patients are unable to connect the Internet temporarily, they are still able to use the telerehabilitation system. When the connection is established again, patients’ motion data can be synchronised to the main database in the server side for further analysis. Indeed when the Internet is not available, patients are unable to receive biofeedback and performance measurement updates if the computational power in their mobile device is insufficient.
4.5. Analysis Oriented Decision Support System
In this subsection, we discuss the concept of analysis oriented decision support system (AODSS), which is a combination of the concept of the serviceoriented decision support system (SODSS) [46] and the clinic decision support system (CDSS) [14]. As the AODSS integrated in the MCC-based telerehabilitation platform, it contributes to the establishment of mobile CDSS mentioned in [14], which was considered a research challenge in CDSS. The contribution of the paper to AODSS is that data is analysed with different granularities, which can be collected from various types of sensors and stored in various databases in the cloud. This method is embedded into the data flow (with green symbols) shown in Figure 4.
The concept of AODSS is illustrated in Figure 6. The first column, including various wearable or nonwearable sensors, shows the source of data, which is not the process of data flow of information query and response between therapist side and the sever side. Data from these sensors are stored in different databases in the cloud which is highly likely to be distributed into different servers to maintain the response speed with a large number of queries. After raising a query from the therapist side (user layer) to the data management layer, the data extraction tool retrieves related data from some databases (in data layer). Then the data is further processed by using various integrated data mining tools (in data analysis layer) and mining algorithms, such as clustering, classification, regression, attribute importance, anomaly detection, association, and feature extraction. Eventually, the data is visually presented to the therapists to assist them to make further decisions.
The key point in this AODSS is that data stored in database is encoded with various encoding scheme. As mentioned above, the selection of the encoding scheme is determined by the computational power of the mobile device and the speed of the Internet. Our AODSS is able to not only analyse data from various types of sensors but also provide supportive information in various granularities to therapists based on the encoding levels of data sources.
Use stroke telerehabilitation as an example. Since the kinematic performance of a poststroke patient can be assessed through the movement of limbs and hands, AODSS first retrieves related data from two databases, including Kinect and Creative Senz3D databases. Due to the fact that data used by AODSS is encoded with various encoding schemes, AODSS is able to provide results in various granularities. For instance, AODSS can provide a histogram showing the change of performance (computed by the performance measurement algorithms mentioned in Section 3.1) of a patient in telerehabilitation sessions during a day, a month, or even longer period. After receiving the analysis results, the therapist will have a general idea about the effectiveness of the therapy and the progress of the patient. If the therapist wants to know the movement patterns of the patient in each session, AODSS is also able to analyse the encoded data at motion trajectory, elbow points, or shape model level and generate a report to show the details in each session. These information assists the therapist to understand the detailed reason why the patient is underprogressing or improving. Thus, the therapists can reevaluate the exercise components in the session to suit the patient’s capabilities to achieve better rehabilitative outcomes or have the confidence to encourage the patient to stick with the therapy. Apart from that, AODSS fuses data from both Kinect and Creative Senz3D to show the performance of the poststroke patient in both limb movements and hand movements. Therefore, the therapist can receive a more comprehensive information and produce better relevant decision for the therapy session.
4.6. Security Service Layer
As mentioned previously, security and privacy are one of the challenges faced in CDSS. Therefore, AODSS is inevitably impacted by this challenge. Security service layer (SSL) introduced in this subsection is a concept that has the potential to solve this challenge in the proposed AODSS, as well as the MCC-based telerehabilitation platform.
Since many parties such as patients, clinicians, healthcare administrators, insurers, and researchers are involved, an end-to-end security control will be applied to enhance the protection of data. The security association is managed by a Security Association Manager to coordinate the communication groups [25]. The Security Association Manager includes Security Policy Collections, Security Association Flows, and Distributed Logs. The Security Policies Collection specifies which security level and which rules need to be used in associations with the types of parties. The Security Association Flows contain routing algorithms, keys, encryption schemes, protocol modes, and flow-level lifetime. The Distributed Logs store the logs of both end points in order to avoid fragment of logs stored in different repository. The Security Association Manager can collaborate with external Certificate Authority [47] to enhance authorization processes. This cooperation allows a patient or the service provider to rely on their identity provider to provide other e-Healthcare providers with only the specific data necessary to complete the transaction.
4.7. Optimize Biofeedback
Traditionally, the feedback with regard to the rehabilitation exercises is given directly from therapists, which is the most effective approach. Therefore, this method is implemented in our system as a biofeedback option. However, it is only available when the therapist reviews the exercises of the patient online so that the auditory message is routed by the server side from the therapist to the selected patient and vice versa for communication. It means that this method relies heavily on the large bandwidth, which is not always available for MCC. Therefore, in our system, we have three other types of biofeedback, which can be selected according to the availability and the bandwidth of the allocated network connection.
First of all, since the model motions recorded by therapists were utilised to guide patients to perform telerehabilitation exercises, patients are able to receive the visual biofeedback directly by looking at the differences between their motion trajectories and those from the model motions. However, due to the fact that the motion trajectories on the screen are always in 2 dimensions and the third one is unable to be observed by patients, the top down view of both the models and patients' motions are presented in the other indow to illustrate the differences in the third dimension. By looking at the screen, patients are able to see the gap between their motions and the models, thereby correcting their motions in time. Since the model motion is always available, either from the cloud (when the speed of the Internet is fast enough) or from the local temporary database (when the bandwidth is small or the Internet is disconnected), this type of biofeedback is always available.
Secondly, auditory biofeedback is also an option in the introduced platform to correct patient’s telerehabilitation exercises. For instance, with the respiratory exercises, music with various rhythms is given so that the patient is able to modify the frequency of the breathing by following the rhythm of the music. The same concept is utilised for motion rehabilitation exercises where fast rhythm indicates that the patient should move the limbs or other body parts involved in the telerehabilitation session faster and the slow rhythm tends to slow down the patient’s movement. To modify the rhythm of the music, the speed of the patient’s movement is derived from their motion trajectories in the server side and is compared with that of the model motion. Eventually, a ratio is generated and sent to the patient side to change the rhythm.
Lastly and most importantly, although we utilise performance measurement as an encoding scheme (refer to Section 3), this value can also be utilised as a feedback indicating the performance of doing specific rehabilitation exercises. As is mentioned in the previous subsection, two approaches (including elbow point based and entropy based) could be utilised to represent the performance of a rehabilitation exercise session. This feedback not only gives both the patient and the therapist an overview about the ability of the patient to perform certain tasks, but also is able to stimulate the patient to perform exercises more frequently, thereby achieving higher performance measurement.
5. Numerical Results and Platform Demonstration
5.1. Computer Simulation for Multilevel Encoding Scheme
In this subsection, a numerical example is presented to demonstrate the effectiveness of the multilevel encoding scheme for motion exercise monitoring in saving energy in mobile devices. First of all, we generated 18 motion trajectories with for each exercise. We further assumed μJ, μJ, μJ, and mJ. These assumptions were based on execution time for each specific task which directly related to the complexity of the computations and transmission. We also assumed that we use a connection setting which cost mJ for transferring data in 1 second. We examined the total energy consumption for both local computations and data transfer for different uploading speeds from 0 kbps to 2000 kbps. The numerical experimental result is shown in Figure 7. In the figure, the minimum energy for the uploading speed from 0 kbps to 50 kbps can be achieved if we use level 4 of encoding scheme. Similarly, we use level 3 for uploading speed from 50 kbps to 200 kbps, level 2 for uploading speed from 200 kbps to 1200 kbps, and level 1 for above 1400 kbps. At the uploading speed of 50 kbps, the total of energy spending for levels is J, J, J, and J. The difference of energy between levels can be up to J and this means the use of encoding schemes can save up to 20 times of energy for low connection speeds. Indeed, for better connections, for example, uploading speed above 1400 kbps, sending all data to the cloud for processing is the best method in terms of saving energy and preserving information.
5.2. Real-Data Experiment for Multilevel Encoding Scheme
To further illustrate the performance of the proposed multilevel encoding scheme, we performed a preliminary real-data experiment with simulated motion trajectory data and four various encoding methods. In this experiment, software named BatteryMon (V2.1 build 1008) and NetBalancer were utilised to monitor the energy consumption and to control the speed of the Internet connection. Furthermore, we implemented the encoding algorithms and data transferring program in C#. The experiment was done on a laptop with CPU of Inter Core i7-3740QM and Wifi card of Intel Centrino Ultimate-N 6300AGN. To eliminate the influence of other software, first of all, BatteryMon was initialised for half an hour without running any unnecessary program to estimate the energy consumption of the laptop in idle state. All the following measurements were subtracted by this energy to compute the power utilised in order to compute the proposed encoding methods or to transfer the data to the cloud.
After that, a 3D trajectory with length of 1000000 frames was collected and further encoded with the other three encoders. Each of the encoding was repeated for 10 times (reliability test) where the average energy consumption of computing for each encoder was recorded in terms of per frame.
Furthermore, the setup of the real-data experiment is shown in Figure 8. We deployed the data receiver component of the data transferring program in a desktop connected to a wireless router with a network cable to secure the speed of the data transfer. Additionally, the laptop is connected to the router with the Wifi card like a normal mobile device. Moreover, NetBalancer was used to control the Internet upload speed of the laptop to simulate the environment with different conditions of the Internet. We limited the upload speed from 80 to 800 Kbps with a step of 80 Kbps for testing because the upload speed on 3G/4G is about 0.45 Mbps to 1.93 Mbps [48]. Eventually, we recorded the average energy utilized to transfer one frame of the encoded data with each respective speed.
Lastly, we computed the total power consumption for each encoding approach and various uploading speeds for a trajectory with 10000 frames. The result is shown in Figure 9.
From the result, a similar trend as the simulation was observed. When the upload speed was smaller than about 160 Kbps, level 4 encoder was the best option since it consumed the lowest energy to encode the motion data and transfer the result to the cloud, while the level 3 encoding approach should be selected when the upload speed of the Internet was ranging from 160 to 480 Kbps and, subsequently, level 2 encoder should be utilised. Using real experiment’s data, it shows that level 1 encoding method always consumes more energy than level 2 and level 3 encoder in this scenario.
5.3. Telerehabilitation Platform Demonstration
This subsection shows snapshots of the implemented telerehabilitation platform. First of all, the example of the model making function in therapist side is shown in Figure 10.
In this figure, it can be seen that a therapist is able to make a specific model for a particular patient according to the condition of this patient. One or multiple joints can be selected as the important joints in this exercise model. Lastly, the functions of replay and crop of the recorded therapist’s model motion were incorporated in the system to adapt the needs of the therapist.
The example of the patient side is shown in Figure 11. There are five components in this figure. The left side of the figure shows the front view of the patient skeleton with the guidance trajectory. The top right of the figure shows the top view of the tracking joint (yellow circle) and the guidance point (light blue circle). The middle right chart shows the history of the performance measurement for a period of time and the meter in the bottom right shows the performance’s level of the current session. Additionally, the message where the Start word appears shows the incoming instructions as a form of visual feedback. The aim of the exercise is to match the red circle on the skeleton onto the red circle on the guidance (blue trajectory).
If the Internet connection is available, the motion trajectories of the patient are encoded with one of the approaches introduced in Section 3 depending on the computational power of the mobile device as well as the upload speed of the Internet. After each session, the performance measurement, which is also the level 4 encoder, is computed either on the local device or in the cloud platform. Eventually, this measurement is shown in the patient side as an overall biofeedback.
6. Conclusion
Optimising the limited battery resources on mobile devices to provide maximum time for patients to access the telerehabilitation services is a major concern in the implementation of a MCC-based telerehabilitation system. This paper introduced a novel multilevel encoding scheme to minimise the energy consumption in the mobile devices by taking the advantage of characteristics of biomedical signals. The computer simulations and experimental data have primarily shown the advantage of the proposed approach in selecting the suitable encoding approach for saving energy.
Apart from that main contribution, the concept of analysis oriented decision support system (AODSS) and a service security layer (SSL) were also introduced as an important component of the proposed telerehabilitation platform. Particularly, the AODSS provides therapists with supportive suggestions by analysing a huge amount of data collected from different types of sensors in various levels of granularities. As a result, more comprehensive results can be supplied to assist therapists to make further decisions. Furthermore, SSL secures the data in collecting, storing, and transferring within the platform.
Different sensor types were successfully tested separately in lab environment with healthy subjects to evaluate the feasibility of overall platform. Future studies will be focused on capture and analysis of clinical data with a larger patient group related to specific medical conditions, such as Parkinson’s disease, stroke, and sleep apnoea, and extending the AODSS database to evaluate the support system. Moreover, various types of new sensors will be added into this system gradually in future clinical trials.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Authors’ Contribution
Saiyi Li and Hai Trieu Pham made equal contributions to this paper.
Acknowledgment
This work is supported by the National Information and Communication Technologies Australia (NICTA) and Deakin University.