In a world with an increasing flow of written information online and offline, the demand for automatic translation, simplification and summarization technology is growing. The tool StilLett uses several lexical and semantic rules to automatically simplify text, shortening the time from original text to simplified text compared to manual simplification. Four of the simplification types were evaluated in this study, using three different eye tracking measures; total dwell time, number of regressions, and average fixation duration. No statistically significant differences were found for any of the simplification types when compared to the corresponding original sentences, indicating that for this population the difficulty of the sentences was roughly the same.