The wildly popular new film A Star Is Born, a tragic musical love story starring Lady Gaga and Bradley Cooper, has revived interest in a somewhat obscure field: dream analysis.

Cooper, who also directed the film, said in a recent New York Times interview that he used his subconscious to make rituals for his character. Cooper and Lady Gaga worked with acting coach Elizabeth Kemp, Cooper’s mentor, who taught him the technique and to whom the film is dedicated, before her death last year.

While not exactly a mainstream practice, dream study goes back at least 3,000 years and has a devoted following today. I was curious, so I decided to give it a go.

Humans have an estimated three to five dreams a night, with Rapid Eye Movement (REM), or dreaming sleep, most commonly falling towards the end of the night. “Dream work” has roots in the techniques of the famed theater actor and director Konstantin Stanislavski and the theories of psychologist Carl Jung, Kim Gillingham, a teacher of the technique, told me.

Dream work is not just for actors, Gillingham said. Having coached film-makers, directors, novelists, dancers and scientists in how to use dreams to find greater authenticity in their work, Gillingham says she has seen dreams’ transformative potential for people across the professional spectrum.

“For one thing, if we have an unresolved trauma or something from childhood, or a pattern that’s inhibiting us, or a pattern that’s drawing us again and again to addictive behaviour, self-loathing behaviour, an old tape running in your head, the dreams will serve up the reality of that for us to work with,” the 55-year-old, who has been in the field since discovering it in her 20s, told me a few days before the workshop.

She also believes that dreams present solutions. “Psychologically, physically, emotionally, I believe wholeheartedly in the comprehensive healing package of what the dream brings for everyone – for the plumber, saint, all of us, have this genius guiding material coming through in our dreams in the night.”

Despite having never done anything like it before, I was fairly relaxed about going to Gillingham’s class – until being asked to sign a waiver accepting responsibility for “the risks to my person and psyche”. Although I’m usually a pretty vivid dreamer, I found the pressure of recording my dreams made remembering them more difficult than normal. But after listening to the trippiest music my boyfriend’s record collection had to offer and avoiding technology before bed, I managed to muster one at the last minute. I scrawled the dream down in as much detail as possible, then hopped on the subway and headed to the class.

Going from the noisy rush of the Manhattan streets to the hushed, almost reverent, anticipation of the softly lit dream studio felt like entering a secret society. People, mainly actors (including some pretty famous ones that I’m not allowed to name), were taping sketches – private images from their unconscious – on to the wall and quietly making notes in their dream journals. At the front was an “altar” decorated with flowers and candles.

Nervously, I took out my notepad, containing some recent dreams and a naive felt-tip pen attempt at drawing last night’s (it involved a chest of drawers, a laptop and, inexplicably, the number 53). I took a seat on the only vacant yoga mat, near the front. Gillingham, who was leading the five-hour dream workshop, gave a short introduction, then we each lit a candle. She rang a horror movie-esque bell that made me jump, marking the start of our journey into the unconscious.

The class was divided into two parts. The first was spent largely with our eyes closed, working individually under Gillingham’s instruction, mentally revisiting the scene of our dream and going to a childhood memory. The second was more like an acting class and revolved around a practical group exercise.

The purpose of the session was to “practice the weaving of inner work and creative work”. Gillingham taught techniques such as “whisper speaking” – quietly uttering thoughts that might be too private or unacceptable to say out loud into cupped hands – and strengthening the “container” (being internally strong enough for the unconscious to come through). She often instructed us to move the position of the tongue and open our mouths to relax the jaw.

Although mentally challenging and emotionally exhausting – at one point it felt so intense that I was nauseous and, like others in the room, I was a couple of times moved to tears – I found the first half most immediately enlightening. The group exercise, where we had to follow our impulses and spontaneously move and make sounds around the room while being mirrored and observed, was in retrospect fascinating but in the moment it felt agonisingly difficult to entirely let go of all inhibitions. There was also an arts and crafts section where we made art with our non-dominant hand and ate snacks.

Do these kinds of exercises actually work? I put the question to Emily Cass McDonnell, a New York-based actor and longtime student of dream work who attended the session. Before she started working with Gillingham seven years ago she hadn’t spent much time dwelling on her dreams. But now McDonnell is a firm convert. She regularly records and explores her dreams, which she said had proved insightful.

“It deepened everything, both my life and my work,” she explained. “There’s a sort of all-knowing or just deeper-knowing benevolent inner resource that you have and it has the language of your dreams. Coming into a relationship with that is for me just really fun and endless.”

McDonnell said her dream work was crucial to her approach to her role last year in Annie Baker’s highly praised off-Broadway play The Antipodes, in which her character regularly peeled and ate a hardboiled egg. Like she would if she had dreamed about an egg, she explored its symbolism and its connotations to give the moment on stage a sense of deeply rooted purpose that became like a “secret ritual”. While she “did not make any big deal out of it”, it became a special moment that people commented on. “It had a richness that had something to it that other people picked up on when it was secret to me and it’s still secret.”

Although she is visibly passionate about the power of dreams, it is also deeply personal; dream work helped her deal with grief after the death of her partner. After workshopping a spider dream with Gillingham, the initially frightening spider became a positive symbol to her. “Really it felt like having an electric shock,” she said. “It was like coming back into life and that made a lot of sense for where I was with my grief.”

Dreams can also play a role in therapy, Louis Hagood, a psychoanalyst and member of the International Association for the Study of Dreams, told me. Like many in the dream world, Hagood, 74, has the blissful countenance of somebody who knows an amazing secret. Seeing a therapist who worked with dreams inspired him to leave the business world midlife and pursue a career in psychoanalysis, he told me over coffee. He says he had a precognitive dream warning him of prostate cancer, which was diagnosed and treated in 2001. “Dreams have not only been a healing force for me, but also a spiritual [force],” he said.

But there is a lot left to discover. Dr Meir Kryger, a leader in sleep medicine, professor at Harvard Medical School and author of The Mystery of Sleep, says that although humans have been dreaming “since day one”, sleep science is still relatively modern. “Rapid Eye Movement sleep, when most vivid dreams appear to occur, was only [discovered] in 1953, and that’s in terms of science pretty modern, so the field is not as new or old as people think it is.”

Kryger is especially interested in research on the relationship between dreaming and learning and the process of how dreams are coded. “In some instances [dreams] can come back over and over and over again, in some cases for 50 years, and we see that in post-traumatic stress disorder,” he said. “That’s something that we don’t really understand in sufficient detail and consequently we can’t really treat it properly.”

If dreams are so important to day-to-day life, then why don’t people pay more attention to them? Dr Rubin Naiman, a psychologist and sleep and dream specialist at the University of Arizona’s Center for Integrative Medicine, believes western society simply does not value sleep as highly as being awake. He calls the problem “wakecentrism”.

Naiman has worked with numerous artists and musicians – including a “world-renowned rock’n’roll group” he cannot name. As well as creativity, dreaming is an essential component of good mental health, he said. It serves as an antidepressant and an anti-inflammatory and is good for memory. “We have a nighttime therapist in our head if we dream well.”

But he warned of the dangers of using technology to try to manipulate dreams by “dream hacking” – a recent interest in monitoring brain waves in order to interrupt sleep to impose lucid dreaming. He called “the notion that we can invade the dream world and do with it what we want” a prime example of “wakecentrism”.

“Watch your dreams tonight,” Gillingham reminded us at the end of her class. She said our unconsciousnesses would want to respond to the work we’d done in the workshop. I couldn’t help but feel a little sceptical. Stepping back into the mayhem of the city, I felt drained and raw.

But the following morning I awoke to find my emotions settled. And, true to promise, I had received a definite – if not immediately decipherable – overnight reply: the chest of drawers was back, this time containing a rabbit.