After the advent of nanotechnology, self-assembly has become an active area of research, as it being one of the few efficient methods to generate ensembles of nanostructures. In this thesis, we present studies on two dimensional self-assembly of polymer grafted nanoparticle (PGNPs) and thin film modelling approach to understand the physics involved in the self-assembly mechanism of polymeric nanoparticles. The two dimensional, hierarchical assemblies of PGNPs are created from evaporating solution films spread at the air-water interface using Langmuir-Blodgett technique. A transition in the patterns is observed with increase in concentration which is followed by a remarkable re-entrance of initial patterns with further concentration increment. The pattern is long length scale network type at low and high concentrations whereas it is short length scale distribution of clusters at intermediate concentrations. Clusters are composed of lateral arrangement of individual PGNPs. The characteristics of clusters are tailored by changing various experimental conditions such as molecular weight of the grafted chains, concentration, temperature and evaporation rate. The patterns are unaffected by the transfer surface pressure, suggesting that the self-assembly occurs in the presence of solvent via solution thin film instabilities and the resulting structures of PGNPs are frozen upon complete evaporation. Films of neat polystyrene also exhibit similar morphology and transitions in pattern length scales with initial solution concentration as observed in PGNP films. This confirms that the self-assembly of PGNPs is driven by the intrinsic nature of the grafted polymer chains. Gradient dynamics model is employed to study the stability and dynamics of polymer solution thin films by incorporating Flory Huggins free energy and concentration dependent Hamaker constant. Dispersion curves obtained from linear stability analysis of thin film equations show existence of bimodal instability in the film that corresponds to dewetting and decomposition. Phase diagram spanned by concentration and Flory parameter indicate that the thin film instability transits from dewetting to decomposition and then re-enters to dewetting with increase in concentration of the solution. Using the material parameters of the PGNP thin films for linear stability analysis, experimental observations of bimodal length scale of patterns and re-entrant nature are well explained. Nonlinear simulations which are performed to capture the evolution of patterns in the film show that the decomposition progresses through different pathways depending upon the concentration of the solution. This is explained by analyzing the local variation of spinodal parameter (curvature of the free energy per unit area) in the film. The gradient dynamics model is extended to study the stability and dynamics of evaporating solution thin films. Nonlinear simulations demonstrate that the film undergoes evaporative thinning without any significant growth of dewetting or decomposition instability initially and becomes unstable at a certain intermediate thickness where the spinodal parameter of dewetting or decomposition changes the sign. The rupture of the film (dewetting) or the phase segregation (decomposition) occurs explosively and subsequently evaporation progresses till the film attains chemical equilibrium with the ambient vapour phase. Rate of evaporation significantly affect the intermediate thickness at which the patterns emerge and thereby determines the length scale of initial patterns and instability growth rate. Quasi-steady analysis and nonlinear simulations show that the length scales of patterns of dewetting and decomposition decrease with evaporation rate and exhibit a power law behaviour. Thin films in which the solvent quality drops down with confinement due to evaporation are modelled by assuming a simple functional dependence of Flory parameter on mean film thickness. Quasi-steady analysis demonstrates that the dominating instability of such films switches from dewetting to decomposition and then returns to dewetting with increase in the initial concentration of the solution. We note that even though the functional form of Flory parameter with confinement is not exact, it represents the essential nature of the expected variation. We presume that the phenomenon discussed above is quite generic and may manifest itself in many situations where thin films of colloidal solutions undergo a decrease in the solvent quality due to confinement effects resulting in a competition between spinodal dewetting and decomposition instabilities. This will result in a competition and interplay of the different instability scales and by choosing appropriate control parameters novel self-assembled patterns can be created.