An intelligent sensor system has the potential of providing its operator with relevant information, lowering the risk of human errors, and easing the operator's workload. One way of creating such a system is by using reinforcement learning, and this thesis studies how reinforcement learning can be applied to a simple sensor control task within a detailed 3D rendered environment. The studied agent controls a stationary camera (pan, tilt, zoom) and has the task of finding stationary targets in its surrounding environment. The agent is end-to-end, meaning that it only uses its sensory input, in this case images, to derive its actions. The aim was to study how an agent using a simple neural network performs on the given task and whether behavior cloning can be used to improve the agent's performance. The best-performing agents in this thesis developed a behavior of rotating until a target came into their view. Then they directed their camera to place the target at the image center. The performance of these agents was not perfect, their movement contained quite a bit of randomness and sometimes they failed their task. But even though the performance was not perfect, the results were positive since the developed behavior would be able to solve the task efficiently given that it is refined. This indicates that the problem is solvable using methods similar to ours. The best agent using behavior cloning performed on par with the best agent that did not use behavior cloning. Therefore, behavior cloning did not lead to improved performance.

No description available.

The following studies describe the synthesis and properties of a family of poly(arylene ether ketone)s which are well known to have good thermal stability, mechanical durability, and other film properties. These poly(arylene ether ketone)s were functionalized with fluorine, oxidized, blended, and crosslinked to increase performance with focus on materials for polymeric capacitors and gas separation membranes. There is a need for polymeric capacitors with improved energy storage density and thermal stability. In this work, the affect of polymer molecular structure and symmetry on Tg, breakdown strength, and relative permittivity was investigated. A systematic series of four amorphous poly(arylene ether ketone)s was compared. Two of the polymers had symmetric bisphenols while the remaining two had asymmetric bisphenols. Two contained trifluoromethyl groups while the other two had methyl groups. The symmetric polymers had Tg's of approximately 160 °C while the asymmetric polymers showed higher Tg's near 180 °C. The symmetric polymers had breakdown strengths near 380 kV/mm at 150 °C. The asymmetric counterparts had breakdown strengths near 520 kV/mm even at 175 °C, with the fluorinated polymers performing slightly better in both cases. The non-fluorinated polymers had higher relative permittivities than the fluorinated materials, with the asymmetric polymers being better in both cases. Two amorphous, high glass transition, crosslinkable poly(arylene ether)s for gas purification membranes have been studied. The polymers were polymerized via step growth and contained tetramethyl bisphenol F and either 4,4'-difluorobenzophenone or 4,4'-dichlorodiphenylsulfone. The benzylic methylene group in tetramethyl bisphenol F can undergo oxidation reactions and crosslinking with UV light. The polymers were oxidized under two different conditions, one by chemical treatment using oxone and KBr and one by elevated thermal treatment in air. Thermogravimetric analysis, 1H-NMR and attenuated total reflectance Fourier transform infrared spectroscopy revealed the progress of the thermal oxidation reactions. Both polymers produced tough, ductile films and gas transport properties of the non-crosslinked linear polymers and crosslinked polymer was compared. Crosslinking was performed by irradiating polymer films for one hour on each side in air under a 100W high intensity, long-wave UV lamp equipped with a 365-nm light filter. The O2 permeability of tetramethyl bisphenol F containing non-crosslinked poly(arylene ether ketone) was 2.8 Barrer, with an O2/N2 selectivity of 5.4. Following UV crosslinking, the O2 permeability decreased to 1.8 Barrer, and the O2/N2 selectivity increased to 6.2. Poly(2,6-dimethyl-1,4-phenylene oxide) (PPO) is a commercial polymer that is utilized for gas separation membranes. It has a relatively high free volume with high gas permeabilities but suffers from low gas selectivities. In this study, PPO polymers with number average molecular weights of 2000, 6000, 17,000, 19,000 and 22,000 were synthesized and blended with a poly(arylene ether ketone) synthesized from bisphenol A and difluorobenzophenone (BPA-PAEK) to make UV-crosslinkable films. The ketone and benzylic methylene groups on the BPA-PAEK and the PPO polymers respectively formed crosslinks upon exposure to broad wavelength UV light. The crosslinked blends had increased selectivities over their linear counterparts. DSC thermograms showed that the blends with all but the lowest molecular weight PPO had two Tg's, thus suggesting that two phases were present, one high in PBA-PAEK and the other high in PPO composition. The PBA-PAEK blend with the 2000 Mn PPO showed only one Tg between the two control polymers. Despite the immiscibility of these films, the gel fractions after UV exposure were high. Gel fractions as a function of the amount of the 22,000 Mn PPO were explored and did not show any significant change. UV spectroscopy of the individual components and the blends showed that more broad wavelength light was transmitted through the PPO component, so it was reasoned that films that was high in PPO composition crosslinked to deeper depths. The O2/N2 permeabilities and selectivities were measured for the linear and crosslinked films. Between the 33/67, 67/33, and 90/10 22k PPO/BPA PAEK crosslinked blended films, the 90/10 PPO/BPA PAEK gained the most selectivity and maintained a larger amount of its permeability. In comparison to commercial gas separation polymers, the non-crosslinked 33/67 22,000 Mn PPO/BPA PAEK blend outperformed polysulfone and cellulose acetate with a 2.45 degree of acetylation. Overall, we were able to blend a small amount of BPA PAEK with the commercially used PPO to create a mechanically robust crosslinked polymer film. / Ph. D.

Given the recent advances within a subfield of machine learning called reinforcement learning, several papers have shown that it is possible to create self-learning digital agents, agents that take actions and pursue strategies in complex environments without any prior knowledge. This thesis investigates the performance of the state-of-the-art reinforcement learning algorithm proximal policy optimization, when trained on a task with nondeterministic state transitions. The agent’s policy was constructed using a convolutional neural network and the game Candy Crush Friends Saga, a single-player match-three tile game, was used as the environment. The purpose of this research was to evaluate if the described agent could achieve a higher win rate than average human performance when playing the game of Candy Crush Friends Saga. The research also analyzed the algorithm's generalization capabilities on this task. The results showed that all trained models perform better than a random policy baseline, thus showing it is possible to use the proximal policy optimization algorithm to learn tasks in an environment with nondeterministic state transitions. It also showed that, given the hyperparameters chosen, it was not able to perform better than average human performance.

Doctor of Philosophy / Polyphenol oxidase (PPO) catalyses undesirable darkening in wheat products such as Asian noodles. Genetic variation for PPO activity is characterized in bread wheat. Australian wheat breeding programmes recognize that reduced PPO activity is an important quality target. Despite this interest from breeders, no varieties possessing extremely low and null PPO activity exist. The development of null PPO wheat varieties is dependant on an understanding of the genetic control of the null phenotype. Knowledge of these factors will accelerate efforts to develop them. The inheritance of PPO activity was investigated in two populations that were derived from hybrids between a null PPO genotype and Australian wheat varieties Lang and QAlBis. Observed genetic ratios were consistent with two and three gene control, respectively in these populations. QTL mapping was performed in the QALBis x VAW08-A17 population. The Diversity Array Technology (DArT) approach was employed to genotype the QALBis x VAW08-A17 population. Three highly significant QTLs that control PPO activity were identified on chromosomes 2AL, 2BS and 2DL. Close associations between PPO activity and DArT marker loci wPt-7024, wPt-0094 and wPt-2544 were observed, respectively. Collectively, these loci explained 74% of the observed variation in PPO activity across seasons. Significant QTLs on chromosomes 1B and 3B were also identified that together explained an additional 17% of variation in PPO activity. The relationship between PPO activity and yellow alkaline noodles (YAN) colour stability parameters was investigated in a DM5637*B8 x H45 doubled haploid population. PPO activity and changes in YAN brightness (ΔL* 0-24h) and yellowness (Δb* 0-24h) in both seasons were analysed. Quantitative trait analyses of PPO activity, flour yellowness (b*) and YAN colour stability was also conducted in this population. QTL mapping of variation in PPO activity in the DM5637*B8 x H45 DH population identified a highly significant QTL on chromosome 2AL, which explained 52% of the observed variation across seasons. Regression analysis identified that wPt-7024 was highly significantly associated with PPO activity in this population. A highly significant association between this marker and PPO was also identified in the QALBis x VAW08-A17 population. Collectively, the three identified QTLs (on chromosomes 2AL, 7A and 7B) explained 71% of variation in PPO activity across seasons. A highly significant (P<0.001) QTL on chromosome 2B along with significant (P<0.01) QTLs on the chromosomes 1A, 3B, 4B and 5B were found to control flour yellowness. The QTLs on 2B, 4B and 5B were detected in both seasons analysed and accounted for 90% of variation in flour b* across seasons. The study on YAN colour stability located two highly significant (P<0.001) QTLs and two significant (P<0.01) QTLs that controlled the change in brightness of yellow alkaline noodle. The 2A QTL accounted for 64% of observed variation across seasons. It was in the same location as the PPO QTL and shared a common closest marker wPt-7024. Only one significant QTL for YAN a* (0-24h) was identified. It accounted for 12% of variation across seasons and was only detected in one season. One highly significant (P<0.001) QTL and two significant (P<0.01) QTLs were identified that controlled the change in yellowness of yellow alkaline noodle. The 2A QTL accounted for 68% of observed variation across seasons. The location of this QTL corresponded with that of 2A QTLs for PPO activity and L* of YAN in this study. Furthermore, wPt-7024 was also identified as the marker with the most significant association with L*. The identification of a correlation between the characters and a common location of a highly significant QTL for each of these characters indicates that it is likely that PPO activity is directly responsible for a large proportion of the changes in brightness and yellowness of YAN. QTLs for L* and b* of YAN were detected in a common location on chromosome 1A. However, no corresponding QTL was identified that controls PPO activity, highlighting the complexity of the relationship between these traits. Resistance to three rust pathogens (Puccinia graminis, Puccinia striiformis, and Puccinia triticina) was also investigated in the DM5637*B8 x H45 DH population because they are major yield limiting diseases in wheat. Disease response data at the seedling stage were converted to genotypic scores for rust genes Sr24/Lr24, Sr36, Lr13 and Yr7 to construct a genetic linkage map. No recombination was observed between rust resistance genes Sr36, Lr13 and Yr7 in this DH population. Therefore, these genes mapped in the same position on chromosome 2B. The Lr24/Sr24 locus was incorporated into the chromosome 3D map. Interval mapping analysis identified QTLs on chromosomes 2B, 3B, 4B and 5B that control adult plant resistance (APR) to stripe rust. Two QTLs on chromosomes 2B and 3D were identified that controlled APR to leaf rust in this DH population.

Doctor of Philosophy / Polyphenol oxidase (PPO) catalyses undesirable darkening in wheat products such as Asian noodles. Genetic variation for PPO activity is characterized in bread wheat. Australian wheat breeding programmes recognize that reduced PPO activity is an important quality target. Despite this interest from breeders, no varieties possessing extremely low and null PPO activity exist. The development of null PPO wheat varieties is dependant on an understanding of the genetic control of the null phenotype. Knowledge of these factors will accelerate efforts to develop them. The inheritance of PPO activity was investigated in two populations that were derived from hybrids between a null PPO genotype and Australian wheat varieties Lang and QAlBis. Observed genetic ratios were consistent with two and three gene control, respectively in these populations. QTL mapping was performed in the QALBis x VAW08-A17 population. The Diversity Array Technology (DArT) approach was employed to genotype the QALBis x VAW08-A17 population. Three highly significant QTLs that control PPO activity were identified on chromosomes 2AL, 2BS and 2DL. Close associations between PPO activity and DArT marker loci wPt-7024, wPt-0094 and wPt-2544 were observed, respectively. Collectively, these loci explained 74% of the observed variation in PPO activity across seasons. Significant QTLs on chromosomes 1B and 3B were also identified that together explained an additional 17% of variation in PPO activity. The relationship between PPO activity and yellow alkaline noodles (YAN) colour stability parameters was investigated in a DM5637*B8 x H45 doubled haploid population. PPO activity and changes in YAN brightness (ΔL* 0-24h) and yellowness (Δb* 0-24h) in both seasons were analysed. Quantitative trait analyses of PPO activity, flour yellowness (b*) and YAN colour stability was also conducted in this population. QTL mapping of variation in PPO activity in the DM5637*B8 x H45 DH population identified a highly significant QTL on chromosome 2AL, which explained 52% of the observed variation across seasons. Regression analysis identified that wPt-7024 was highly significantly associated with PPO activity in this population. A highly significant association between this marker and PPO was also identified in the QALBis x VAW08-A17 population. Collectively, the three identified QTLs (on chromosomes 2AL, 7A and 7B) explained 71% of variation in PPO activity across seasons. A highly significant (P<0.001) QTL on chromosome 2B along with significant (P<0.01) QTLs on the chromosomes 1A, 3B, 4B and 5B were found to control flour yellowness. The QTLs on 2B, 4B and 5B were detected in both seasons analysed and accounted for 90% of variation in flour b* across seasons. The study on YAN colour stability located two highly significant (P<0.001) QTLs and two significant (P<0.01) QTLs that controlled the change in brightness of yellow alkaline noodle. The 2A QTL accounted for 64% of observed variation across seasons. It was in the same location as the PPO QTL and shared a common closest marker wPt-7024. Only one significant QTL for YAN a* (0-24h) was identified. It accounted for 12% of variation across seasons and was only detected in one season. One highly significant (P<0.001) QTL and two significant (P<0.01) QTLs were identified that controlled the change in yellowness of yellow alkaline noodle. The 2A QTL accounted for 68% of observed variation across seasons. The location of this QTL corresponded with that of 2A QTLs for PPO activity and L* of YAN in this study. Furthermore, wPt-7024 was also identified as the marker with the most significant association with L*. The identification of a correlation between the characters and a common location of a highly significant QTL for each of these characters indicates that it is likely that PPO activity is directly responsible for a large proportion of the changes in brightness and yellowness of YAN. QTLs for L* and b* of YAN were detected in a common location on chromosome 1A. However, no corresponding QTL was identified that controls PPO activity, highlighting the complexity of the relationship between these traits. Resistance to three rust pathogens (Puccinia graminis, Puccinia striiformis, and Puccinia triticina) was also investigated in the DM5637*B8 x H45 DH population because they are major yield limiting diseases in wheat. Disease response data at the seedling stage were converted to genotypic scores for rust genes Sr24/Lr24, Sr36, Lr13 and Yr7 to construct a genetic linkage map. No recombination was observed between rust resistance genes Sr36, Lr13 and Yr7 in this DH population. Therefore, these genes mapped in the same position on chromosome 2B. The Lr24/Sr24 locus was incorporated into the chromosome 3D map. Interval mapping analysis identified QTLs on chromosomes 2B, 3B, 4B and 5B that control adult plant resistance (APR) to stripe rust. Two QTLs on chromosomes 2B and 3D were identified that controlled APR to leaf rust in this DH population.

This study is focused on study of changes of enzyme and low-molecular weight antioxidants in different fruits during long-term storage. In theoretical part individual low-molecular weight antioxidants and enzymes are described. The main causes of fruit decay and some possibilities of fruit preservation and storage are summarized. As biological material some common fruits were chosen - green and red apples, peaches, plums and white grapes. The fruits were stored in laboratory, cellar, in refrigerator and in freezer. In freezing experiments some ways of fruit preparation and processing were tested and their influence on fruit antioxidant status was compared. Shortened storage experiment was applied on blueberries, cranberries, raspberries and strawberries too. In fruits some group parameters – total antioxidant status, dry mass content, ascorbate level, total flavonoids and total phenolics were analyzed spectrophotometrically. Individual flavonoids and phenolics were determined by RP-HPLC/UV-VIS and on-line LC/PDA/ESI-MS. Antioxidant enzyme activities (superoxide dismutase SOD, catalase CAT, polyphenol oxidase PPO and lipoxygenase LOX) were measured by spectrophotometry. The surface microscopy and cultivation of moulds from fruit surface were performed too. Influence of storage conditions on biological activities is dependent on fruit sort. Freezing is the most suitable procedure for long-term storage without significant changes of active substance content. Long-term storage in controlled temperature conditions and/or atmosphere is usable for fruits with longer storage period. In these fruits stabile levels of antioxidant enzymes are stored for relatively long time. Some of enzymes act synergistically. Enzyme activities differed according to storage phase; at the beginning mainly high SOD and LOX activities were observed. CAT and PPO are probably activated as defence systems in rippened and/or damaged fruits. Levels of total as well as individual low molecular weight antioxidants varied during storage in all sorts, generally, increased course with longer storage period can be observed.

This thesis explores the use of reinforcement learning within the Unity ML Agents framework to simulate zombie-like movements in humanoid ragdolls. The generated locomotion aims to embrace the Uncanny Valley phenomenon, partly through the way it walks, but also through limb disablement. Additionally, the paper strives to test the effectiveness of reinforcement learning as a valuable tool for generative adaptive locomotion. The research implements reward functions and addresses technical challenges. It lays a focus on adaptability through the limb disablement system. A user study comparing the reinforcement learning agent to Mixamo animations evaluates the effectiveness of simulating zombie-like movements as well as if the Uncanny Valley phenomenon was achieved. Results show that while the reinforcement learning agent may lack believability and uncanniness when compared to the Mixamo animation, it features a level of adaptability that is worth expanding upon. Given the inconclusive results, there is room for further research on the topic to achieve the Uncanny Valley effect and enhance zombie-like locomotion with reinforcement learning.

Multi-agent path planning is applied in a wide range of applications in robotics and autonomous vehicles, including aerial vehicles such as drones and other unmanned aerial vehicles (UAVs), to solve tasks in areas like surveillance, search and rescue, and transportation. In today's rapidly evolving technology in the fields of automation and artificial intelligence, multi-agent path planning is growing increasingly more relevant. The main problems encountered in multi-agent path planning are collision avoidance with other agents, obstacle evasion, and pathfinding from a starting point to an endpoint. In this project, the objectives were to create intelligent agents capable of navigating through two-dimensional eight-agent cost map environments to a static target, while avoiding collisions with other agents and simultaneously minimizing the path cost. The method of reinforcement learning was used by utilizing the development platform Unity and the open-source ML-Agents toolkit that enables the development of intelligent agents with reinforcement learning inside Unity. Perlin Noise was used to generate the cost maps. The reinforcement learning algorithm Proximal Policy Optimization was used to train the agents. The training was structured as a curriculum with two lessons, the first lesson was designed to teach the agents to reach the target, without colliding with other agents or moving out of bounds. The second lesson was designed to teach the agents to minimize the path cost. The project successfully achieved its objectives, which could be determined from visual inspection and by comparing the final model with a baseline model. The baseline model was trained only to reach the target while avoiding collisions, without minimizing the path cost. A comparison of the models showed that the final model outperformed the baseline model, reaching an average of $27.6\%$ lower path cost. / Multi-agent-vägsökning används inom en rad olika tillämpningar inom robotik och autonoma fordon, inklusive flygfarkoster såsom drönare och andra obemannade flygfarkoster (UAV), för att lösa uppgifter inom områden som övervakning, sök- och räddningsinsatser samt transport. I dagens snabbt utvecklande teknik inom automation och artificiell intelligens blir multi-agent-vägsökning allt mer relevant. De huvudsakliga problemen som stöts på inom multi-agent-vägsökning är kollisioner med andra agenter, undvikande av hinder och vägsökning från en startpunkt till en slutpunkt. I detta projekt var målen att skapa intelligenta agenter som kan navigera genom tvådimensionella åtta-agents kostnadskartmiljöer till ett statiskt mål, samtidigt som de undviker kollisioner med andra agenter och minimerar vägkostnaden. Metoden förstärkningsinlärning användes genom att utnyttja utvecklingsplattformen Unity och Unitys open-source ML-Agents toolkit, som möjliggör utveckling av intelligenta agenter med förstärkningsinlärning inuti Unity. Perlin Brus användes för att generera kostnadskartorna. Förstärkningsinlärningsalgoritmen Proximal Policy Optimization användes för att träna agenterna. Träningen strukturerades som en läroplan med två lektioner, den första lektionen var utformad för att lära agenterna att nå målet, utan att kollidera med andra agenter eller röra sig utanför gränserna. Den andra lektionen var utformad för att lära agenterna att minimera vägkostnaden. Projektet uppnådde framgångsrikt sina mål, vilket kunde fastställas genom visuell inspektion och genom att jämföra den slutliga modellen med en basmodell. Basmodellen tränades endast för att nå målet och undvika kollisioner, utan att minimera vägen kostnaden. En jämförelse av modellerna visade att den slutliga modellen överträffade baslinjemodellen, och uppnådde en genomsnittlig $27,6\%$ lägre vägkostnad.

京都大学 / 0048 / 新制・課程博士 / 博士(教育学) / 甲第18727号 / 教博第165号 / 新制||教||149(附属図書館) / 31678 / 京都大学大学院教育学研究科教育科学専攻 / (主査)教授 田中 耕治, 准教授 西岡 加名恵, 准教授 服部 憲児 / 学位規則第4条第1項該当 / Doctor of Philosophy (Education) / Kyoto University / DGAM