2021

1. Healthinf

   Convolutional Neural Network for Elderly Wandering Prediction in Indoor Scenarios

   Oliveira, Rafael F, Barreto, Fabio, and Abreu, Raphael

   In Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5 HEALTHINF: HEALTHINF 2021

   resumo arXiv Code Video

   This work proposes a way to detect the wandering activity of Alzheimer’s patients from path data collected from non-intrusive indoor sensors around the house. Due to the lack of adequate data, we’ve manually generated a dataset of 220 paths using our own developed application. Wandering patterns in the literature are normally identified by visual features (such as loops or random movement), thus our dataset was transformed into images and augmented. Convolutional layers were used on the neural network model since they tend to have good results finding patterns, especially on images. The Convolutional Neural Network model was trained with the generated data and achieved an f1 score (relation between precision and recall) of 75%, recall of 60%, and precision of 100% on our 10 sample validation slice

2020

1. IEEE MultiMedia

   Towards content-driven intelligent authoring of mulsemedia applications

   Abreu, Raphael, Mattos, Douglas, Santos, Joel, Ghinea, Gheorghita, and Muchaluat-Saade, Débora

   IEEE MultiMedia 2020

   resumo PDF Video

   Synchronization of sensory effects with multimedia content is a non-trivial and error-prone task that can discourage authoring of mulsemedia applications. Although there are authoring tools that perform some automatic authoring of sensory effect metadata, the analysis techniques that they use are not general enough to identify complex components that may be related to sensory effects. In this work, we present a new method, which allows the semi-automatic definition of sensory effects in an authoring tool. We outline a software component to be integrated into authoring tools that uses content analysis assistance to indicate moments of sensory effects activation, according to author preferences. The proposed method was implemented in the STEVE 2.0 authoring tool and an evaluation was performed to assess the precision of the generated sensory effects in comparison with human authoring. This solution is …

2. SBCAS

   Ambientes multissensoriais aplicados à saúde: desenvolvimento de aplicações e tendências futuras

   Ivanov, Marina, Montevecchi, Eyre, Abreu, Raphael, Barreto, Fabio, Santos, Joel, and Muchaluat-Saade, Débora Christina

   Sociedade Brasileira de Computação 2020

   HTML

2019

1. IEEE Transactions on Learning Technologies

   Supporting the Learning of Evolution Theory Using an Educational Simulator

   Cardoso, Josué, Caetano, Diego, Abreu, Raphael, Quadros, João, Santos, Joel, Ogasawara, Eduardo, and Lignani, Leonardo

   IEEE Transactions on Learning Technologies 2019

   PDF
2. WebMedia

   NCL: Storing user settings in media node

   Barreto, Fábio, Montevecchi, Eyre Brasil B, Abreu, Raphael, Santos, Joel AF, and Muchaluat-Saade, Débora C

   In Anais Estendidos do XXV Simpósio Brasileiro de Sistemas Multimı́dia e Web 2019
3. WebMedia

   Providing multi-user in NCL with userAgent and UserProfile

   Barreto, Fábio, Montevecchi, Eyre Brasil B, Abreu, Raphael, Santos, Joel AF, and Muchaluat-Saade, Debora C

   In Anais Estendidos do XXV Simpósio Brasileiro de Sistemas Multimı́dia e Web 2019
4. WebMedia

   Mais de um sentido: Facilitando a autoria, sincronização e execução de efeitos sensoriais em linguagens multimı́dia

   Abreu, Raphael, Bezerra, Eduardo, and Santos, Joel

   In Anais Estendidos do XXV Simpósio Brasileiro de Sistemas Multimı́dia e Web 2019
5. WebMedia

   A Proposal for Supporting Sensory Effect Rendering in Ginga-NCL

   Oliveira Rodrigues, Renato, Josué, Marina IP, Abreu, Raphael S, Amorim, Glauco F, Muchaluat-Saade, Debora C, and Santos, Joel AF

   In Anais Estendidos do XXV Simpósio Brasileiro de Sistemas Multimı́dia e Web 2019
6. WebMedia

   A proposal for supporting sensory effect rendering in ginga-ncl

   Rodrigues, Renato O, Josué, Marina IP, Abreu, Raphael S, Amorim, Glauco F, Muchaluat-Saade, Débora C, and Santos, Joel AF dos

   In Proceedings of the 25th Brazillian Symposium on Multimedia and the Web 2019
7. WebMedia

   Authoring sensory effects in NCL

   Barreto, Fábio, Abreu, Raphael S, Santos, Joel AF, and Muchaluat-Saade, Débora C

   In Anais Estendidos do XXV Simpósio Brasileiro de Sistemas Multimı́dia e Web 2019
8. WebMedia

   Providing Multimodal User Interaction in NCL

   Barreto, Fábio, Montevecchi, Eyre Brasil B, Abreu, Raphael, Santos, Joel AF, and Muchaluat-Saade, Debora C

   In Anais Estendidos do XXV Simpósio Brasileiro de Sistemas Multimı́dia e Web 2019
9. WebMedia

   Semi-automatic synchronization of sensory effects in mulsemedia authoring tools

   Abreu, Raphael, Mattos, Douglas, Santos, Joel AF dos, and Muchaluat-Saade, Débora C

   In Proceedings of the 25th Brazillian Symposium on Multimedia and the Web 2019

   HTML
10. WebMedia

    NCL: Storing ambient settings in media node

    Barreto, Fabio, Montevecchi, Eyre Brasil B, Abreu, Raphael, Santos, Joel AF, and Muchaluat-Saade, Débora C

    In Anais Estendidos do XXV Simpósio Brasileiro de Sistemas Multimı́dia e Web 2019
11. LANOMS

    Latency-Aware Placement of Software Defined Network Controllers

    Abreu, Raphael, Albuquerque, Célio, and Muchaluat-Saade, Débora

    In IX Latin-American Network Operations and Management Symposium (LANOMS 2019) 2019

    HTML

2018

1. IJCNN

   A Bimodal Learning Approach to Assist Multi-sensory Effects Synchronization

   Abreu, Raphael, Santos, Joel, and Bezerra, Eduardo

   In International Joint Conference on Neural Networks 2018

   resumo arXiv Code Video

   In mulsemedia applications, traditional media content (text, image, audio, video, etc.) can be related to media objects that target other human senses (e.g., smell, haptics, taste). Such applications aim at bridging the virtual and real worlds through sensors and actuators. Actuators are responsible for the execution of sensory effects (e.g., wind, heat, light), which produce sensory stimulations on the users. In these applications sensory stimulation must happen in a timely manner regarding the other traditional media content being presented. For example, at the moment in which an explosion is presented in the audiovisual content, it may be adequate to activate actuators that produce heat and light. It is common to use some declarative multimedia authoring language to relate the timestamp in which each media object is to be presented to the execution of some sensory effect. One problem in this setting is that the synchronization of media objects and sensory effects is done manually by the author(s) of the application, a process which is time-consuming and error prone. In this paper, we present a bimodal neural network architecture to assist the synchronization task in mulsemedia applications. Our approach is based on the idea that audio and video signals can be used simultaneously to identify the timestamps in which some sensory effect should be executed. Our learning architecture combines audio and video signals for the prediction of scene components. For evaluation purposes, we construct a dataset based on Google’s AudioSet. We provide experiments to validate our bimodal architecture. Our results show that the bimodal approach produces better results when compared to several variants of unimodal architectures.

2. MMSYS

   Modeling sensory effects as first-class entities in multimedia applications

   Josué, Marina, Abreu, Raphael, Barreto, Fábio, Mattos, Douglas, Amorim, Glauco, Santos, Joel, and Muchaluat-Saade, Débora

   In Proceedings of the 9th ACM Multimedia Systems Conference 2018

   HTML

2017

1. DocEng

   Using Abstract Anchors to Aid The Development of Multimedia Applications With Sensory Effects

   Abreu, Raphael, and Santos, Joel AF

   In ACM Symposium on Document Engineering 2017

   HTML
2. WebMedia

   Using Abstract Anchors for Automatic Authoring of Sensory Effects Based on Ambient Sound Recognition

   Abreu, Raphael, and Santos, Joel

   In Brazilian Symposium on Multimedia and the Web 2017

   HTML

2016

1. ICCMSE

   Simulation step size analysis of a whole-cell computational model of bacteria

   Abreu, Raphael, Castro, Maria Clicia S, and Silva, Fabrício Alves B

   17th International Conference of Computational Methods in Sciences and Engineering 2016

   HTML