“Far-out” is an accurate, but inadequate, descriptor for the high-flying (and often plain high) cultural magus named Harry Smith (1923-91). And the label “polymath,” too, while true, falls short for this innovative painter-filmmaker-collagist-musicologist-designer-scholar-curator-collector/hoarder, whose very first and very strange (it could not be otherwise) institutional solo is at the Whitney Museum of American Art.

When speaking of Smith, it’s hard to know where to begin, or end. To the degree that he is familiar at all in the art world (never mind in the real world) it’s as an experimental filmmaker. His chief reputation, however, lies in a different field, music, notably as the compiler of the 1952 six-LP collection called the “Anthology of American Folk Music,” an ethnological document that had a subtle but palpable role in moving the nation’s sociopolitical needle in a revolutionary direction during the civil rights and Vietnam era.

How to present such a figure, whose work is so grounded in sound and visual motion, in a traditional museum setting naturally presents a problem, which the Whitney has handily solved by bringing in an object-based artist, the sculptor Carol Bove, as installation designer.

Bove has created a big, film-friendly, black-box-style container for the show. And she has placed down at its center a zigzagging walled corridor for the display of little-known objects — paintings, drawings, prints, photographs — that Smith produced almost nonstop throughout his life and that he sometimes claimed to regard more highly than his films.

That life began in the Pacific Northwest. Smith was born in Portland, Ore., and grew up in Washington State. He was lucky in his family. They didn’t have money: His father worked in the fish-canning industry; his mother was a teacher. But they encouraged his early interest in reading and art and folk music. And as practicing Theosophists, they made him comfortable with esoteric spiritualities and instilled in him their own pantheistic love of the natural world.

Because his mother taught school on the local Lummi Indian reservation, Smith became fascinated with Indigenous culture. By age 15, he was already a committed ethnologist, participating in Lummi dances and religious rituals, absorbing Native music, photographing objects, sacred and secular — a handful of foggy slide photographs of masks, drums and weavings are the show’s earliest entries — while taking copious field notes on everything.

And a unitary concept of “Everything” was already the axis around which his worldview turned. He was intensely focused — a classic geek — but the focus was panoramic and panoptic, taking in many seemingly unalike things — dance, color, language — at once, all of which he perceived as interrelated. He would speak of illuminating such connection as the primary value of his work, the one he cared most about.

In 1945, he moved to San Francisco with the intention of studying anthropology at the University of California in Berkeley. But classroom learning wasn’t his thing. (He attended some lectures but never registered.) He spent most of his time doing what amounted to field research in the city’s burgeoning Beat poetry cafes and in jazz clubs where Dizzy Gillespie and Charlie Parker regularly played.

He lived in a minute apartment in the Fillmore neighborhood, then predominantly African American, and indulged what would be two insatiable lifelong appetites: one, for mood-altering substances (alcohol and a rainbow of perception-changing drugs), and the other for the bulk collecting of objects — books, music recordings, artworks (for him a spacious, nonhierarchical category), antique tools, tarot cards, textiles, toys, used bandages found at tattoo shops, and a Himalaya of newspaper and magazine clippings.

In San Francisco he was doing a lot of painting: smoothly geometric Kandinsky-ish, mandalalike compositions, as well as looser, brushier work in which the individual strokes were synced to the notes and chords in jazz recordings. And he used this gestural mode to create his first animated abstractions, painted directly on film stock, which was then edited and projected.

The earliest surviving example of this “action painting,” “Film No. 1: A Strange Dream” (circa 1946-48), is in the show — it’s an eyepopper — as are a few more abstractions from the San Francisco years. They’re tip-of-the-iceberg evidence of the riches Smith was producing at the time. But they also hint at what’s been lost.

Chronically indigent and often high, Smith was careless with his art and collections. When he couldn’t pay rent he’d be out on the street, his possessions with him, up for grabs. He’d sometimes destroy things in a rage. So, materially speaking, there’s now relatively little output to see. Three beautiful “jazz paintings” in the show exist only as lightbox transparencies made from slides of originals lost who knows when. As a result, a show of big ideas — organized by Dan Byers of the Carpenter Center for the Visual Arts at Harvard; Rani Singh, director of the Harry Smith Archives; and Elisabeth Sussman, Kelly Long and McClain Groff of the Whitney — feels small.

Smith was blessed with protective friends — the poet Allen Ginsberg and the filmmaker Jonas Mekas were two — and sporadically with supportive patrons, including, briefly, Hilla Rebay, the first director of the Museum of Non-Objective Painting (the forerunner of the Guggenheim Museum).

On a visit to San Francisco in 1948 she saw Smith’s extraordinary animated abstractions and offered him a stipend to do more. With the money he moved to New York City, settling first on the Lower East Side, and later and longer, in the Chelsea Hotel on West 23rd Street. Here he worked on some of his most ambitious projects.

In 1952, the Manhattan-based Folkway Records released his “Anthology of American Folk Music,” the long-time-coming end product of Smith’s childhood passion for preserving materials from sources he perceived as marginalized. And although the LP set had a low-key landing — it was niche marketed, primarily to libraries — it gained a passionate and eclectic audience that included Bob Dylan, Philip Glass and the Grateful Dead.

(The full “Anthology” set, which Smith regarded as an art object in itself — he even signed it as if it were a painting — can be sampled in a section of the show set aside as a listening station, as can the fabulously erudite and poetic commentary that Smith wrote for all 84 cuts.)

In New York, he also created his most complex and inventive films, none of them, strictly speaking, abstract. “Film No. 11: Mirror Animations,” made around 1957, adheres to the “jazz painting” model of aligning music and visuals. The music in this case is Thelonious Monk’s “Misterioso,” but the images now include Buddhist figures and Kabbalistic emblems.

For “Film No. 12: Heaven and Earth Magic Feature,” also in the show, Smith supplied his own score of everyday noises: dogs barking, babies crying, wind blowing, glass breaking. He also proposed a story line — a woman with a toothache goes to a dentist, gets injected with some kind of drug and ascends to heaven — which is enacted by figures clipped from Victorian-era print sources.

The ingenious animation feels delightfully witty at first, but over the span of its hour length, makes for creepy watching. There’s wild, violent stuff going on. If this is heaven, we want to stay clear. Smith has a reputation for being an occultist, but he was never a religionist. Like Joseph Cornell, he was an uninnocent mystic. However spacey his art, the world is very much in it.

It’s certainly there in the magnum opus “Film No. 18: Mahagonny,” (1970-80). The score is a full two-hours-plus recording of the Kurt Weill-Bertolt Brecht opera “The Rise and Fall of the City of Mahagonny.” And the visuals, projected on four square contiguous screens, are a collage of color films Smith shot in Manhattan in the 1970s: on its streets, in the Chelsea Hotel and in Central Park.

A mathematically calculated visual puzzle, it’s also a record of a time and place, filtered through Smith’s favored themes: outsider-insider culture, embodied in figures from the city’s avant-garde (Ginsberg and Patti Smith make appearances); material accumulation (tabletop arrangements of food, liquor bottles and drugs); and some promise of transcendence, in this case through Nature (childhood: he keeps going back there).

In the 1970s, New York was in trouble, and so was Smith. Years of alcohol and drug intake were catching up. “A stoned, drunken, hunched-over demonically creative gnome” is how his New York psychiatrist described him. Penniless and in failing health, he was crashing with friends who passed him on to other friends. At one point he ended up in a Bowery flophouse. (This phase of his life — indeed his entire life — is empathetically chronicled in John Szwed’s indispensable new biography, “Cosmic Scholar: The Life and Times of Harry Smith.”)

But he never stopped working, which meant collecting: He carried a tape recorder, always turned on. And there were late upbeat moments. In 1988 he was invited to teach at Naropa Institute (now Naropa University) in Boulder, Colo., a Buddhist-inspired college, where he was treasured and cosseted.

In 1991, he was awarded a special Grammy for the “Anthology” and flew to New York, five kittens in tow, to accept it. He wore a rented tuxedo. No one would have guessed that by this point he was surviving entirely on instant mashed potatoes, NyQuil and cigarettes and would soon be lost in hallucinations of who he would meet in the afterlife. He died, at the Chelsea Hotel, that year, “unique, devious, saintly,” as Ginsberg eulogized, and far-out right to the end.

Fragments of a Faith Forgotten: The Art of Harry Smith

Through Jan. 28, 2024, at the Whitney Museum of American Art, 99 Gansevoort Street, Manhattan; 212-570-3600, whitney.org.