The study of waves and instabilities in quantum plasmas is of fundamental importance for understanding collective interactions in superdense astrophysical objects, in high intense laser-plasma/solid-matter interactions, in microelectronic devices and metallic nanostructures. In dense quantum plasmas, there are new pressure laws associated with the Fermi-Dirac distribution functions and new quantum forces associated with the quantum Bohm potential and the Bohr magnetization involving electron ½ spin. These forces significantly alter the collective behavior of dense quantum plasmas. This thesis contains six papers, considering several novel collective modes and instabilities at quantum scales. In Paper I, we have used the quantum hydrodynamical (QHD) model for studying the one-dimensional dust-acoustic (DA) waves incorporating the Fermi pressure law and the quantum Bohm potential. The latter modifies the DA wave dispersion relation in a collisional plasma. In Paper II, we have calculated the electrostatic potential of a test charge in an unmagnetized electron-ion quantum plasma. It is found that the Debye-Hückel and oscillatory wake potentials strongly depend upon the Fermi energy at quantum scales. The results can be of interest for explaining the charged particle attraction and repulsion in degenerate quantum plasmas, such as those in semiconductor and microelectronic devices. Paper III presents the parametric study of nonlinear electrostatic waves in two-dimensional collisionless quantum dusty plasmas. A reductive perturbation method has been employed to the QHD equations together with the Poisson equation, obtaining the cylindrical Kadomtsev-Petviashvili (CKP) equations and their stationary localized solutions. We have numerically examined the quantum mechanical and geometrical effects on the profiles of nonplanar quantum dust-ion-acoustic (DIA) and DA solitary waves. The role of static as well as mobile (negatively or positively charged) dust particles on the low-frequency electrostatic waves has also been highlighted for metallic nanostructures. Paper IV introduces the nonlinear properties of the ion-sound waves in a dense electron-ion Fermi magnetoplasma. The computational analysis of the nonlinear system reveals that the Sagdeev-like potential and the ion-sound density excitations are significantly affected by the wave direction cosine and the Mach number at quantum scales. Paper V considers the nonlinear interactions of electrostatic upper-hybrid (UH), ion-cyclotron (IC), lower-hybrid (LH), and Alfvén waves in a quantum magnetoplasma. The nonlinear dispersion relations have been analyzed analytically to obtain the growth rates for both the decay and modulational instabilities involving the dispersive IC, LH, and Alfvén waves. In Paper VI, we have identified a new drift-like dissipative instability in a collisional quantum plasma. The modified unstable drift-like mode can cause cross-field anomalous ion-diffusion at quantum scales.

In the Thesis, two important theoretical problems arising in the theory of one-dimensional defocusing nonlinear Schrödinger (NLS) flows are investigated analytically and numerically: (i) the resonant generation of dispersive shock waves (DSWs) in one-dimensional NLS flow past a broad repulsive penetrable barrier; and (ii) the interaction of counter-propagating DSW and a simple rarefaction wave (RW), which is referred to as the DSW refraction problem. The first problem is motivated by the recent experimental observations of dark soliton radiation in a cigar-shaped BEC by sweeping through it a localised repulsive potential; the second problem represents a dispersive-hydrodynamic counterpart of the classical gas-dynamics problem of the shock wave refraction on a RW, and, apart from its theoretical significance could also find applications in superfluid dynamics. Both problems also naturally arise in nonlinear optics, where the NLS equation is a standard mathematical model and the `superfluid dynamics of light' can be used for an all-optical modelling of BEC flows. The main results of the Thesis are as follows: (i) In the problem of the transcritical flow of a BEC through a wide repulsive penetrable barrier an asymptotic analytical description of the arising wave pattern is developed using the combination of the localised ``hydraulic'' solution of the 1D Gross-Pitaevskii (GP) equation with repulsion (the defocusing NLS equation with an added external potential) and the appropriate exact solutions of the Whitham-NLS modulation equations describing the resolution of the upstream and downstream discontinuities through DSWs. We show that the downstream DSW effectively represents the train of dark solitons, which can be associated with the excitations observed experimentally by Engels and Atherton (2008). (ii) The refraction of a DSW due to its head-on collision with the centred RW is considered in the frameworks of two one-dimensional defocusing NLS models: the standard cubic NLS equation and the NLS equation with saturable nonlinearity, the latter being a standard model for the light propagation through photorefractive optical crystals. For the cubic nonlinearity case we present a full asymptotic description of the DSW refraction by constructing appropriate exact solutions of the Whitham modulation equations in Riemann invariants. For the NLS equation with saturable nonlinearity, whose modulation system does not possess Riemann invariants, we take advantage of the recently developed method for the DSW description in non-integrable dispersive systems to obtain key parameters of the DSW refraction. In both problems, we undertake a detailed analysis of the flow structure for different parametric regimes and calculate physical quantities characterising the output flows in terms of relevant input parameters. Our modulation theory analytical results are supported by direct numerical simulations of the corresponding full dispersive initial value problems (IVP).