Action recognition had become prosperous in development and been broadly applied in several sectors. From homeland security, personal property, home caring, even the smart environment and the motion-sensing games, are in its territories This paper analysis the algorithm of Action recognition for embedded system, finds that there are many blocks can use the parallel execution to compute more efficiently. This paper tries to implement action recognition algorithm on Multiple-Streaming Multimedia Unit (MSMU). MSMU is a MMX-like SIMD architecture, with SIMD Operation and Data Storage. By introduction the concept of multiple streaming, MSMU will be able to modulate the amount of parallel data streams dynamically via switching the instruction mode. With Mode Switching and new added transfer instruction to compute 2D image processing, study the benefit of the instruction mode switching Through comparing the 128-bit SSE architecture and MSMU architecture with the practical example, highlight the problems that exploiting the subword parallelisms facing and bring out the advantage of Multistreaming. For the algorithm, study the slicing the minimum element and using the bitwise operation approach to better efficiency. Compare to embedded SIMD architecture "WMMX", MSMU can achieve 3.49¡Ñ overall speedup.