National Lab (LBNL) results support LK-99 as a room-temperature ambient-pressure superconductor.
— Andrew Cote (@Andercot) August 1, 2023
Simulations published 1 hour ago on arxiv support LK-99 as the holy grail of modern material science and applied physics.
(https://t.co/4t4D2gIeBp)
Here's the plain-english… pic.twitter.com/mQNQuO4TFu
Kaguya is a ChatGPT plugin that allows you to load and edit your local files in a controlled way, as well as run any Python, JavaScript, and bash script. This makes it a powerful tool for developers, enabling them to interact with their file system and run scripts directly from ChatGPT. To interact with Kaguya, you'll need access to plugin devtools by getting on their waitlist here. In case this approach doesn't work for you, we may be able to come up with a more open approach at some point in the future.
Here are a few demo videos of Kaguya:
docker.sh scriptThe project provides several API endpoints that allow you to interact with the file system. The API is described in the openapi.yaml file. Here is a brief overview:
GET /api/listFilesInDirectory: List files and directories in the specified directory. Defaults to FILES.GET /api/readFile: Read the content of a file in the user's directory.GET /api/readMultipleFiles: Read the content of multiple files.POST /api/update: Update a file in the user's directory by performing a search-and-replace operation.POST /api/updateAll: Update a file in the user's directory by performing a search-and-replace operation (all occurrences).POST /api/updateWholeFile: Replace the entire content of a file in the user's directory.POST /api/appendToFile: Append content to the end of an existing file.POST /api/createFile: Create a new file.POST /api/deleteFile: Delete a file in the user's directory.POST /api/renameFile: Rename a file in the user's directory.POST /api/createDirectory: Create a new directory.POST /api/deleteDirectory: Delete a directory and its contents.POST /api/executeCommand: Execute a shell command.Feel free to join our Discord server here.
Secure your supply chain and ship with confidence
The Socket Security browser extension adds security metrics to your NPM package pages and search results, protecting you from threats in open-source packages before you even install them. By the time CVEs and known vulnerabilities make it to public databases, it's often too late. Using advanced code analysis techniques and AI-powered risk detection, Socket searches for malware and security vulnerabilities throughout your open-source dependency tree and defends your project against cyberattacks in advance. --- Over the past decade, it's become clear that open source software has won. Sharing code freely has made it drastically cheaper and faster to build software – and tech innovation has accelerated as a result. But security has often been an afterthought. We are a team of open source maintainers with over 1 billion monthly downloads to our names. Working on the frontlines of open source, we've witnessed firsthand how supply chain attacks have swept across our communities and damaged trust in open source. The entire security industry is obsessed with identifying known vulnerabilities. There are hundreds of variations of CVE scanners, but they all miss the point. Looking for known vulnerabilities is reactive. Vulnerabilities take weeks or months to be discovered. In today's culture of fast development, a malicious dependency can be updated, merged, and running in production in days or even hours. Unlike other tools, Socket detects and blocks supply chain attacks before they strike, mitigating the worst consequences. Socket uses deep package inspection to peel back the layers of a dependency to characterize its actual behavior. Want to defend your entire organization against open-source attacks? Install the Socket GitHub app at https://github.com/apps/socket-security and get protected today!
It’s been a while since we’ve seen a .4 release, but quite a few people have reported a crash when a news item appears while a drop-down menu is open.
Here’s a fix for that, plus a few other fixes completed since the last release.
Following the attacks of September 11th, Congress passed the Aviation and Transportation Security Act (ATSA), creating the Transportation Security Agency (TSA). The TSA replaced private security screening companies with one government agency. Since then, air travelers have bowed to pat downs, bans on water bottles and other inconvenient, intrusive procedures as the “new normal” at our nation’s airports. But does any of this make us safer?
Security expert Bruce Schneier coined the term “security theater” to describe some of the TSA’s procedures and screening practices. Security theater provides the appearance of enhanced security without actually making anyone more secure.
Since 9/11, the TSA has implemented new screening procedures on an almost constant basis. The structural problem with these new screening procedures is two-fold. First, these procedures are almost always in response to past threats, not in anticipation of future threats. Second, average Americans suffer the consequences for years to come in the form of ever-increasing screening procedures and lost time.
Sadly, the TSA’s accumulated procedures and screening practices are actually causing more American deaths. Cornell University researchers found decreased air travel after 9/11 led to an extra 242 road fatalities per month. In all, the researchers estimate that 1,200 people died as a result of decreased air travel. In 2007, the Cornell researchers studied TSA screening procedures implemented in 2002, and found that they decreased air travel by 6% – leading to an additional 129 road fatalities in the last three months of 2002. In terms of casualties, that’s the same as blowing up a fully loaded Boeing 737.
There have been significant and numerous changes in the TSA’s security theater since 9/11. Here are some of the highlights and lowlights:
Nowadays, the TSA runs background checks on all travelers before they even arrive at the airport. Using the Department of Homeland Security’s Automated Targeting System, a massive database that employs algorithms to identify potential terrorists, the TSA automatically places thousands of travelers on the so-called selectee list, which earns them an “SSSS” stamp on their boarding pass and extra screening every time they fly. For travelers who want to avoid extra screening, there’s the TSA’s PreCheck program, which lets passengers keep their shoes, belts and jackets on and skip lengthy lines in exchange for a background check, fingerprinting and a fee.
Although security procedures have gotten more aggressive under the TSA, detection rates seem no better than they were before September 11, 2001. An undercover investigation by the DHS in 2018, found that the TSA had equipment or procedure failure more than half the time.
Numbers don’t lie. The TSA’s failure rate at weapon detection remains strong, which is likely80% at some major airports. And during undercover tests, that failure rate increases. During covert tests conducted by the DHS in 2015, TSA agents failed to detect guns and fake explosives 95 percent of the time. In one test, an undercover DHS agent was stopped and received an “enhanced” pat-down search after setting off a metal detector, but the TSA screener failed to detect the fake bomb taped to the agent’s back.
The U.S. hasn’t suffered any major attacks since 9/11. However, incidents like shoe bomber Richard Reid weren’t thwarted by the TSA – they were stopped by watchful passengers.
Since 2006, the TSA has spent more than $1 billion on training so-called Behavior Detection Officers (BDOs) who watch passengers for suspicious behavior so they can be singled out for extra screening at airports. The program is called Screening Passengers by Observation Techniques (SPOT).
The specific behaviors that BDOs are trained to look for are supposed to be secret, but a leaked copy of the behavior checklist includes fidgeting, sweating, yawning and “suspicious” eye movements. SPOT operates on a point system. Sweaty palms and whistling earns travelers one point. Arrogance, a cold stare and good posture earn travelers two points. Other indicators include a recently shaved face, looking at the floor, clearing your throat, and complaining about the screening process. Travelers who get enough points can be flagged for extra screening.
Since its inception in 2006, and after spending more than $1 billion, the SPOT program has only identified one man who could be called a terrorist. Kevin Brown, a Jamaican citizen, was waiting for his flight back home at Orlando International Airport in 2008. He checked his two bags, but not before a BDO flagged him as suspicious and ordered extra screening of his checked luggage – which contained pipes, caps, fuses, fuel and a handy printout explaining how to put them all together to make a bomb.
Fewer than one out of the 30,000 passengers who are flagged every year as “suspicious” and are searched, end up getting arrested (usually for carrying drugs, undeclared currency or having outstanding warrants or being in the country illegally). A report by the Government Accountability Office in 2013 surveyed more than 400 scientific studies and found no evidence that terrorists can be identified by the behaviors on the SPOT checklist, and recommended that the TSA cut funding for SPOT and stop training new BDOs.
Israel faces more terrorist threats than any other country in the world, yet their airport security is a model of sensible, measured response to a very real danger.
The screening process at Ben Gurion International Airport starts with a vehicle and bag inspection before passengers even enter the airport. Once inside the airport, passengers must pass through increasing layers of security, facing heavier and heavier scrutiny until they board. The level of technology at Ben Gurion hasn’t changed much since the 1980s. Metal detectors, X-ray scanners and bomb-sniffing dogs are still considered state-of-the-art.
Israeli security screeners approach passengers to ask about their travel plans as standard practice. If passengers’ answers seem practiced or implausible, security personnel can order extra screening. Instead of a checklist, Israeli security screeners rely on intuition and the art of conversation to judge passengers’ intentions. Science bears this out as a more effective technique than a behavior checklist.
Under the rules of the SPOT program, a BDO may see a passenger they feel is suspicious. However, the BDO cannot order extra screening unless the passenger displays approved “suspicious” behaviors like clearing their throat or looking at the floor. Since BDOs are supposed to remain undercover, they cannot approach passengers to gauge their intentions through conversation. That forces American airport security to rely on body language alone, which is actually worse than chance at identifying liars and others with concealed intent.
There are three problems with the Israeli screening procedures, however: First, Israelis in general are used to higher levels of security scrutiny as part of their overall culture. This means both a greater acceptance of the idea of profiling and police interaction, but also a greater deal of confidence and trust in security and police forces.
Perhaps more importantly, there are potential Constitutional issues. Profiling – racial or otherwise – does not currently violate Fourth Amendment protections, but no one is eager to test this in a court of law. Still, as proponents of the Israeli system are quick to point out, their profiling is behavioral, relying less upon unsuccessful pseudoscience and more upon a general “feel” officers trained in behavioral sciences get from interacting directly with travelers.
Finally, there are concerns that the Israeli system simply won’t scale. Still, it’s worth asking how the Israeli system applied to the United States could get any worse.
The TSA’s poor performance record has led some airports back to the private screening companies that handled security before 9/11. Under the TSA’sScreening Partnership Program, an approval process that typically takes years, airports can hire private security screeners. The only requirement is that the private outfits maintain the same level of security as the TSA. More than 20 local and international airports have joined the program – with San Francisco and Kansas City already onboard and more set to join as the TSA promises waiting times will reach three hours at busy airports during the spring and summer travel seasons.
Private screeners at SPP airports have proven themselves to be more efficient and more effective than the TSA. A report by a House oversight committee in 2013 found that private screeners at San Francisco International Airport were much better at detecting prohibited items than TSA screeners at LAX, and wait times were shorter. As a result of the report, calls are growing in Congress to abolish the TSA and return to private screening companies.
“The TSA and Security Theater: Understanding American Airport Security Following 9/11” originally appeared in the Resistance Library on Ammo.com.
Sequoia founder Don Valentine would ask founders two questions: “why now?” and “so what?” At the heart of these questions is the combination of curiosity and rigor that asks what has changed in the world (why now?) and what will this mean (so what?).
AI has a compelling “why now?” with the development of large language models (LLMs) trained with the Transformer architecture. Transformers are well-suited to GPUs, making it practical to marshall immense amounts of data and compute to train AI models with billions and trillions of parameters.
We also now have a persuasive “so what?” These technologies have enabled a whole new user interface for computers: human language. Just like the graphical user interface made the personal computer accessible to millions of customers in the 1980s, so too have the new natural language interfaces made AI accessible to hundreds of millions of users worldwide in the past year.
Artificial intelligence has endured many up and down cycles. When AI is in ascent, as it is now, the term is overused to include not only the leading edge of predictive technology but also any software that is in some sense “smart.” During previous AI Winters, however, researchers retreated to safer terms like machine learning.
John McCarthy coined the term AI in the 1950s to distinguish his research from his older rival Norbert Wiener’s cybernetics. Yet McCarthy himself became disenchanted with the term, complaining, “As soon as it works, no one calls it AI anymore.” McCarthy called the tendency of people to rename past AI efforts with more functional descriptions as soon as they were sufficiently solved the “AI effect” and it affects us to this day. The history of AI is littered with accomplishments that have worked well enough to no longer be considered sufficiently intelligent to earn the aspirational monniker.
As a quick refresher, consider computer vision which underpins current advances in image generation. For a long time, detecting objects in images or videos was cutting edge AI, now it’s just one of many technologies that allow you to order an autonomous vehicle ride from Waymo in San Francisco. We no longer call it AI. Soon we’ll just call it a car. Similarly, object detection on ImageNet was a major breakthrough of deep learning in 2012 and is now on every smartphone. No longer AI.
On the natural language side, there’s a long history before ChatGPT, Claude and Bard burst on the scene. I remember using Dragon Speech-To-Text circa 2002 to type emails when I had a broken arm. What was once called AI is now just “dictation” and is on every phone and computer. Language translation and sentiment analysis, once hard problems in NLP, are now mostly table stakes. Not AI.
It’s also easy to forget how much of what we take for granted on the cloud emerged from previous AI disciplines like recommendation systems (Netflix and Amazon) and path optimization (Google Maps and UPS). The more everyday a thing becomes, the less likely we are to call it AI.
What is considered AI and what is not is important to founders because in the long run it’s always better to underpromise and overdeliver. In what Gartner has described over decades of technological hype cycles, the wild enthusiasm is invariably followed by disappointment—the trough of disillusionment.
Founders benefit in the short term from the buzzy marketing, but at a cost. Arthur C. Clark famously wrote, “Any sufficiently advanced technology is indistinguishable from magic.” But he was a science fiction writer. Machine Learning practitioners are scientists and engineers and yet at first blush their efforts always appear to be magic—until one day it’s not.
The current AI paradigm is like chasing a carrot on a stick while running on a treadmill. For today’s founders I think it’s time to break this cycle by understanding what’s really going on.
There is a linguistic reason we keep making the same mistakes. If we use the Oxford English Dictionary to recursively dissect the term “Artificial Intelligence” we find:
Each branch of this recursive definition hinges on either “human beings” or “a person.” So by definition we think of AI as imitative of humans. Think of the Turing test. But as soon as a capacity is firmly in the realm of machines, we lose the human reference point and we cease to think of it as AI.
Part of this is human exceptionalism. Over the centuries we have elevated the aspects of intelligence that seem uniquely human: language, imagination, creativity and logic. We reserve certain words for ourselves. Humans think and reason, computers calculate. Humans make art, computers generate it. Humans swim, boats and submarines do not. And yet “computer” was once a 17th century job title for a human that calculated, and we employed rooms full of them before we formalized mechanical and electronic computers.
The AI effect is actually part of a larger human phenomenon we call the frontier paradox. Because we ascribe to humans the frontier beyond our technological mastery, that frontier will always be ill-defined. Intelligence is not a thing that we can capture but an ever-approaching horizon that we turn into useful tools. Technology is the artifice of intelligence forged over millennia of human collaboration and competition.
Back in 2018, I was inspired by a post from Berkeley Statistics and Computer Science professor Michael Jordan. “Whether or not we come to understand ‘intelligence’ any time soon,” he wrote, “we do have a major challenge on our hands in bringing together computers and humans in ways that enhance human life. While this challenge is viewed by some as subservient to the creation of ‘artificial intelligence,’ it can also be viewed more prosaically—but with no less reverence—as the creation of a new branch of engineering.”
This led me to write my own post questioning the usefulness of calling this endeavor AI at all. Five years later, are we any closer to Jordan’s vision of a practical infrastructure for human augmentation? I believe we are, but we need a more precise vocabulary to harness the computational opportunity ahead.
The amazing effectiveness of LLMs to generate coherent and believable language has taken almost everyone by surprise. The ability of diffusion models to generate highly-detailed and aesthetically appealing images from text descriptions has also surpassed conventional assumptions. And there’s much more on the horizon in terms of further improvements in language and images, generalization to video, and new innovations in robotics, autonomous vehicles, biology, chemistry and medicine.
All of these advances benefit from the infrastructure for distributed computing that the last wave of hyper-scaled tech companies built in the cloud. They also benefit from the sheer scale of data that has accumulated on the internet, particularly thanks to the ubiquity of highly usable mobile devices with their cameras, sensors and ease of data entry.
But calling all of these things AI confuses the public and founders about what really needs to be built and how to bring it all together in safe and moral ways that encourage both experimentation and responsible behavior.
Given all of this amazing infrastructure, AI as a science project at the intersection of computer science, physics, cognitive and neuroscience will surely advance along the frontier of understanding. It will continue to contribute useful applications as well, but if we call all of them AI, the term will quickly lose its meaning and its novelty.
Experts estimate that even with information retrieval, LLMs are accurate ~90% of the time. There is still a lot of research and scaling to get to 99%. However, once they reach that 99% they will no longer be AI, they will be “language interfaces” or simply LLMs. You will be able to write code on the fly, communicate with people in other languages, learn or teach anything we are interested in, and more. The impact will be real. But we will not call it AI. These new capabilities will become invisible to us, additional parts of our extended minds along with our search engines and smartphones.
This is the frontier paradox in action. AI is accelerating so quickly that it will soon simply be technology and a new frontier will be AI. Graduating to technology should be seen as a badge of honor for an idea that previously was on the cutting edge of possible. The frontier paradox means AI will perpetually refer to aspirational approaches, while technology will refer to what can be put to work today. It is our belief that we need both.
At Sequoia we have tried to become more precise about how we discuss AI internally and with founders. We focus on specific technologies that can be put to work, like transformers for large language models or diffusion for image generation. This makes our ability to evaluate a venture much more explicit, tangible and real.
The entrepreneurial journey starts with language. It is only through language that companies can express the uniqueness of their product and its benefit to customers long before it is ready to ship. The precision of language is the key to category creation, company design and market leadership—the components that make enduring companies.
This precision is even more important as founders surf the rising waves of AI to stay right on the frontier. The founders who can define language as this frontier turns into everyday technology will have a distinct advantage.
If you’re founding a company pushing the frontier of AI into useful technology, we’d love to hear from you. Ambitious founders can increase their odds of success by applying to Arc, our catalyst for pre-seed and seed stage companies. Applications for Arc America Fall ’23 are now open—apply here.
A few weeks ago, a very critical vulnerability allowing arbitrary remote code execution on clients and servers (and therefor even all connected clients on a server) was discovered in many Minecraft mods.
Initially we were trying to investigate the whole issue privately and responsible so we can publish an extensive writeup and fix about the whole situation but since a group named MMPA just published a blog post about the issue, completely missing many important factors about the issue, we were forced to release a statement and attempt to fix the issue immediately since at the current time they're literally putting millions of modded Minecraft users at risk.
The vulnerability is caused by an unsafe use of the Java serialization feature in network packets sent by servers to clients or clients to servers that allows to instantiate any Java class that is loaded in the Minecraft instance.
There was already a similar vulnerability in the past called "Mad Gadget". You can read more about that here:
While there are just a relatively small amount of attacks targetting this vulnerability in the wild, because of the significance of the vulnerability, it is completely dangerous to play with unpatched mods currently. Attackers already attempted (and succeeded in some cases) Microsoft access token and browser session steals. But since they can literally execute any code they want on a target system, the possibilities are endless.
We developed a patcher that attempts to fix all currently known affected mods (listed below).
Should any more affected mods be discovered, a patch is as simple as updating the related config file. (We will publish a relesae that automates this for you) Version 1.3 of the patch now automatically uses the the latest version of the config file and otherwise falls back to the local config file. If there's no config present, there should be an error informing the user that there are currently no patches applied.
-javaagent:<PATH TO SAVED JAR FILE>Unlike stated in the above blog post, there are plenty more mods that are affected by this issue. Although some of them already are fixed in the latest versions, these mods were exploitable in at least one older version:
KEEP IN MIND THAT THIS LIST IS DEFINITELY NOT COMPLETE. THESE ARE JUST THE MODS WE ARE CURRENTLY AWARE OF. At least Curseforge is already investigating the issue internally so we can maybe get a nearly complete list of vulnerable mods and versions in the future.
Because of the rushed announcement, we are currently unable to give exact version ranges of affected mods. If you want to help out with that, feel free to contribute to this list.
I'm not the only one that was working on the investigation of the whole situation.
Credits to anyone that was involved in this:
A blue iceberg is visible after the ice from above the water melts, causing the smooth portion of ice from below the water to overturn.[1][2] The rare blue ice is formed from the compression of pure snow, which then develops into glacial ice.[3][4]
Icebergs may also appear blue due to light refraction and age. Older icebergs reveal vivid hues of green and blue, resulting from a high concentration of color, microorganisms, and compacted ice.[5] One of the better known blue icebergs rests in the waters off Sermilik fjord near Greenland. It is described as an electric blue iceberg and is known to locals as "blue diamond".[6]
Commonly seen white icebergs generally derive their color from the snow and frost remaining on the surface which results in the uniform reflection of incident light. Young glaciers that have not undergone years of compression, may also appear white. Due to the age of the iceberg, there remains a tremendous amount of air and reflective surfaces. The iceberg easily reflects the sun as white light.[7]
Blue icebergs develop from older, deep glaciers which have undergone tremendous pressure experienced for hundreds of years. The process releases and eliminates air that was originally caught in the ice by falling snow. Therefore, icebergs that have been formed from older glaciers have little internal air or reflective surfaces. When long wavelength light (i.e. red) from the sun hits the iceberg, it is absorbed rather than reflected. The light transmitted or refracted through the ice returns as blue or blue-green. Older glaciers also reflect incident light preferentially at the short wavelength end of the spectrum (i.e. blue) due to Rayleigh scattering, much in the same way that makes the sky blue.[7]
Light is absorbed and reflected in water. Visible white light is made up of a spectrum of colors from the rainbow, ranging from red to violet. As the light travels through the water, the waves of light from the red end of the spectrum dissipate (i.e. are absorbed), while those from the blue end, become more prominent.[8]
Underwater divers have direct experience of these effects. Above the water, all the colors remain visible. As the diver swims deeper under water, the colors begin to disappear, starting with red. At an approximate depth of 30 feet (9.1 m), red is no longer visible to the naked eye. At 75 feet (23 m), yellow looks greenish-blue, because the water has absorbed the yellow light. Finally, all that remains visible to the naked eye, appears as a mutation of blue or green, while the water above the surface filters out the sunlight. As the diver swims deeper into the ocean, he finds that the blue colors start to disappear, to the point where the underwater world deep below the surface, becomes completely black, devoid of any color at all.[8][9]
Since 1912, reports made by witnesses of the RMS Titanic tragedy have stated that the ship hit a blue iceberg.[10] Following the sinking and subsequent discovery of the Titanic, scientific research and forensic analysis have reconstructed the tragedy to ascertain the reliability of the statements made by the survivors. Reports released in the last decade of the 20th century have shown that a blue iceberg in the north Atlantic would have been easily detected.[2]Alternative theories suggest that pack ice, rather than a blue iceberg, was responsible for sinking the ship.[11][12]
OpenTelemetry and Prometheus are two critical projects in the monitoring and observability world. Prometheus supports metrics monitoring, and OpenTelemetry also allows metrics besides logs and traces.
The semantic convention for metrics in OpenTelemetry(OTLP metrics) does not align with Prometheus' native metrics naming convention.
To address this disparity, there is a module in otel-collector-contrib that offers centralized functions that facilitate the conversion of OpenTelemetry metrics into metrics compliant with Prometheus.
This package translates and maps metric names, units, and labels between the OpenTelemetry and Prometheus conventions. This translation allows sending OTLP metrics to Prometheus using an OpenTelemetry collector. But the native support for ingesting OTLP metrics was absent in Prometheus.
Recently a pull request was merged in the Prometheus codebase, which adds support for ingesting OpenTelmetry metrics using a new OTLP-compatible ingestion endpoint.
A new feature flag otlp-write-receiver has been added, enabling the feature for natively ingesting OpenTelemetry metrics.
The OpenTelemetry metrics can be sent on /otlp/v1/metrics the endpoint and ingested natively.
This unlocks an alternative strategy to push the O
This change is still experimental, and before the final release, a lot of documentation updates will also happen; I will update this post with the changes and add the final Prometheus release version in which this change will be present!
In the last few months, we have witnessed the rapid progress of the open-source ecosystem for LLMs — from the original LLaMA model that triggered the “LLaMA moment”, to efforts such as RedPajama, MPT, Falcon, and the recent LLaMA-2 release, open-source models have been catching up with closed-source models. We believe the upcoming opportunity for open-source models is to extend the context length of open models to the regime of 32K-128K, matching that of state-of-the-art closed-source models. We have already seen some exciting efforts here such as MPT-7B-8K and LLongMA-2 (8K).
Today, we’re sharing with the community some recent learnings and explorations at Together AI in the direction of building long-context models with high quality and efficiency. Specifically:
LLaMA-2-7B-32K: We extend LLaMA-2-7B to 32K long context, using Meta’s recipe of interpolation and continued pre-training. We share our current data recipe, consisting of a mixture of long context pre-training and instruction tuning data.
Examples of building your own long-context models: We share two examples of how to fine-tune LLaMA-2-7B-32K to build specific applications, including book summarization and long-context question answering.
Software support: We updated both the inference and training stack to allow for efficient inference and fine-tuning with 32K context, using the recently released FlashAttention-2 and a range of other optimizations. This allows one to create their own 32K context model and conduct inference efficiently.
Try it yourself:
Go to Together API and run LLaMA-2-7B-32K for inference.
Use OpenChatKit to fine-tune a 32K model over LLaMA-2-7B-32K for your own long context applications.
Go to HuggingFace and try out LLaMA-2-7B-32K.
Long-context models are already crucial for document understanding, summarization, and retrieval augmented generation. We are excited to share this work with the open-source community and make sustained progress towards better, longer-context models.
LLaMA-2 has a context length of 4K tokens. To extend it to 32K context, three things need to come together: modeling, data, and system optimizations.
On the modeling side, we follow Meta’s recent paper and use linear interpolation to extend the context length. This provides a powerful way to extend the context length for models with rotary positional embeddings. We take the LLaMA-2 checkpoint, and continue pre-training/fine-tuning it with linear interpolation for 1.5B tokens.
But this alone is not enough. What data should we use in improving the base model? Instead of simply fine-tuning using generic language datasets such as Pile and RedPajama as in Meta’s recent recipe, we realize that there are two important factors here and we have to be careful about both. First, we need generic long-context language data for the model to learn how to handle the interpolated positional embeddings; and second, we need instruction data to encourage the models to actually take advantagement of the information in the long context. Having both seems to be the key.
Our current data recipe consists of the following mixture of data:
In the first phase of continued pre-training, our data mixture contains 25% RedPajama Book, 25% RedPajama ArXiv (including abstracts), 25% other data from RedPajama, and 25% from the UL2 Oscar Data, which is a part of OIG (Open-Instruction-Generalist), asking the model to fill in missing chunks, or complete the text. To enhance the long-context capabilities, we exclude sequences shorter than 2K tokens. The UL2 Oscar Data encourages the model to model long-range dependencies.
We then fine-tune the model to focus on its few shot capacity with long contexts, including 20% Natural Instructions (NI), 20% Public Pool of Prompts (P3), 20% the Pile. To mitigate forgetting, we further incorporate 20% RedPajama Book and 20% RedPajama ArXiv with abstracts. We decontaminated all data against HELM core scenarios (see a precise protocol here). We teach the model to leverage the in-context examples by packing as many examples as possible into one 32K-token sequence.
We evaluate the model in two ways: (1) its normalized perplexity under various sequence lengths on PG-19, and (2) its HELM v1.0 scores over 16 core scenarios (evaluated on the same context length that fits LLaMA 2). We see that LLaMA-2-7B-32K incurs reasonable perplexity, comparable to the original LLaMA 2 model. Moreover, on HELM v1.0, LLaMA-2-7B-32K achieves comparable, if not better, quality against the original LLaMA-2-7B base model.
LPython is a Python compiler that can compile type-annotated Python code to optimized machine code. LPython offers several backends such as LLVM, C, C++, WASM, Julia and x86. LPython features quick compilation and runtime performance, as we show in the benchmarks in this blog. LPython also offers Just-In-Time (JIT) compilation and seamless interoperability with CPython.
We are releasing an alpha version of LPython, meaning it is expected you encounter bugs when you use it (please report them!). You can install it using Conda (conda install -c conda-forge lpython), or build from source.
Based on the novel Abstract Semantic Representation (ASR) shared with LFortran, LPython’s intermediate optimizations are independent of the backends and frontends. The two compilers, LPython and LFortran, share all benefits of improvements at the ASR level. “Speed” is the chief tenet of the LPython project. Our objective is to produce a compiler that both runs exceptionally fast and generates exceptionally fast code.
In this blog, we describe features of LPython including Ahead-of-Time (AoT) compilation, JIT compilation, and interoperability with CPython. We also showcase LPython’s performance against its competitors such as Numba and C++ via several benchmarks.
LPython ships with the following backends, which emit final translations of the user’s input code:
LPython can simultaneously generate code into multiple backends from its Abstract Semantic Representation (ASR) of user code.
First, input code is transformed into an Abstract Syntax Tree (AST) using parsers. The AST is then transformed into an Abstract Semantic Representation (ASR), which preserves all semantic information present in the input code. ASR contains all information required by all backends in a form that is not specific to any particular backend. Then, this ASR enjoys several ASR-to-ASR passes, wherein abstract operations are transformed into concrete statements. For example, array addition in the input code denoted, c = a + b. The front end transforms c = a + b into the ASR (Assign c (ArrayAdd a b)) via operator overloading. The array_op ASR-to-ASR pass transforms (Assign c (ArrayAdd a b)) into loops:
for i0 in range(0, length_dim_0):
for i1 in range(0, length_dim_1):
....
....
c[i0, i1, ...] = a[i0, i1, ...] + b[i0, i1, ...]
After applying all the ASR-to-ASR passes, LPython sends the final ASR to the backends selected by the user, via command-line arguments like, --show-c (generates C code), --show-llvm (generates LLVM code).
One can also see the generated C or LLVM code using the following
from lpython import i32
def main():
x: i32
x = (2+3)*5
print(x)
main()
$ lpython examples/expr2.py --show-c
#include <inttypes.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#include <lfortran_intrinsics.h>
void main0();
void __main____global_statements();
// Implementations
void main0()
{
int32_t x;
x = (2 + 3)*5;
printf("%d\n", x);
}
void __main____global_statements()
{
main0();
}
int main(int argc, char* argv[])
{
_lpython_set_argv(argc, argv);
__main____global_statements();
return 0;
}
$ lpython examples/expr2.py --show-llvm
; ModuleID = 'LFortran'
source_filename = "LFortran"
@0 = private unnamed_addr constant [2 x i8] c" \00", align 1
@1 = private unnamed_addr constant [2 x i8] c"\0A\00", align 1
@2 = private unnamed_addr constant [5 x i8] c"%d%s\00", align 1
define void @__module___main_____main____global_statements() {
.entry:
call void @__module___main___main0()
br label %return
return: ; preds = %.entry
ret void
}
define void @__module___main___main0() {
.entry:
%x = alloca i32, align 4
store i32 25, i32* %x, align 4
%0 = load i32, i32* %x, align 4
call void (i8*, ...) @_lfortran_printf(i8* getelementptr inbounds ([5 x i8], [5 x i8]* @2, i32 0, i32 0), i32 %0, i8* getelementptr inbounds ([2 x i8], [2 x i8]* @1, i32 0, i32 0))
br label %return
return: ; preds = %.entry
ret void
}
declare void @_lfortran_printf(i8*, ...)
define i32 @main(i32 %0, i8** %1) {
.entry:
call void @_lpython_set_argv(i32 %0, i8** %1)
call void @__module___main_____main____global_statements()
ret i32 0
}
declare void @_lpython_set_argv(i32, i8**)
LPython implements several machine-independent optimisations via ASR-to-ASR passes. Some of those are listed below,
All optimizations are applied via one command-line argument, --fast. To select individual optimizations instead, write a command-line argument like the following:
--pass=inline_function_calls,loop_unroll
Following is an examples of ASR and transformed ASR after applying the optimisations
from lpython import i32
def compute_x() -> i32:
return (2 * 3) ** 1 + 2
def main():
x: i32 = compute_x()
print(x)
main()
$ lpython examples/expr2.py --show-asr
(TranslationUnit
(SymbolTable
1
{
__main__:
(Module
(SymbolTable
2
{
__main____global_statements:
(Function
(SymbolTable
5
{
})
__main____global_statements
(FunctionType
[]
()
Source
Implementation
()
.false.
.false.
.false.
.false.
.false.
[]
[]
.false.
)
[main]
[]
[(SubroutineCall
2 main
()
[]
()
)]
()
Public
.false.
.false.
()
),
compute_x:
(Function
(SymbolTable
3
{
_lpython_return_variable:
(Variable
3
_lpython_return_variable
[]
ReturnVar
()
()
Default
(Integer 4)
()
Source
Public
Required
.false.
)
})
compute_x
(FunctionType
[]
(Integer 4)
Source
Implementation
()
.false.
.false.
.false.
.false.
.false.
[]
[]
.false.
)
[]
[]
[(=
(Var 3 _lpython_return_variable)
(IntegerBinOp
(IntegerBinOp
(IntegerBinOp
(IntegerConstant 2 (Integer 4))
Mul
(IntegerConstant 3 (Integer 4))
(Integer 4)
(IntegerConstant 6 (Integer 4))
)
Pow
(IntegerConstant 1 (Integer 4))
(Integer 4)
(IntegerConstant 6 (Integer 4))
)
Add
(IntegerConstant 2 (Integer 4))
(Integer 4)
(IntegerConstant 8 (Integer 4))
)
()
)
(Return)]
(Var 3 _lpython_return_variable)
Public
.false.
.false.
()
),
main:
(Function
(SymbolTable
4
{
x:
(Variable
4
x
[]
Local
()
()
Default
(Integer 4)
()
Source
Public
Required
.false.
)
})
main
(FunctionType
[]
()
Source
Implementation
()
.false.
.false.
.false.
.false.
.false.
[]
[]
.false.
)
[compute_x]
[]
[(=
(Var 4 x)
(FunctionCall
2 compute_x
()
[]
(Integer 4)
()
()
)
()
)
(Print
()
[(Var 4 x)]
()
()
)]
()
Public
.false.
.false.
()
)
})
__main__
[]
.false.
.false.
),
main_program:
(Program
(SymbolTable
6
{
__main____global_statements:
(ExternalSymbol
6
__main____global_statements
2 __main____global_statements
__main__
[]
__main____global_statements
Public
)
})
main_program
[__main__]
[(SubroutineCall
6 __main____global_statements
2 __main____global_statements
[]
()
)]
)
})
[]
)
$ lpython examples/expr2.py --show-asr --pass=inline_function_calls,unused_functions
(TranslationUnit
(SymbolTable
1
{
__main__:
(Module
(SymbolTable
2
{
__main____global_statements:
(Function
(SymbolTable
5
{
})
__main____global_statements
(FunctionType
[]
()
Source
Implementation
()
.false.
.false.
.false.
.false.
.false.
[]
[]
.false.
)
[main]
[]
[(SubroutineCall
2 main
()
[]
()
)]
()
Public
.false.
.false.
()
),
main:
(Function
(SymbolTable
4
{
_lpython_return_variable_compute_x:
(Variable
4
_lpython_return_variable_compute_x
[]
Local
()
()
Default
(Integer 4)
()
Source
Public
Required
.false.
),
x:
(Variable
4
x
[]
Local
()
()
Default
(Integer 4)
()
Source
Public
Required
.false.
),
~empty_block:
(Block
(SymbolTable
7
{
})
~empty_block
[]
)
})
main
(FunctionType
[]
()
Source
Implementation
()
.false.
.false.
.false.
.false.
.false.
[]
[]
.false.
)
[]
[]
[(=
(Var 4 _lpython_return_variable_compute_x)
(IntegerBinOp
(IntegerBinOp
(IntegerBinOp
(IntegerConstant 2 (Integer 4))
Mul
(IntegerConstant 3 (Integer 4))
(Integer 4)
(IntegerConstant 6 (Integer 4))
)
Pow
(IntegerConstant 1 (Integer 4))
(Integer 4)
(IntegerConstant 6 (Integer 4))
)
Add
(IntegerConstant 2 (Integer 4))
(Integer 4)
(IntegerConstant 8 (Integer 4))
)
()
)
(GoTo
1
__1
)
(BlockCall
1
4 ~empty_block
)
(=
(Var 4 x)
(Var 4 _lpython_return_variable_compute_x)
()
)
(Print
()
[(Var 4 x)]
()
()
)]
()
Public
.false.
.false.
()
)
})
__main__
[]
.false.
.false.
),
main_program:
(Program
(SymbolTable
6
{
__main____global_statements:
(ExternalSymbol
6
__main____global_statements
2 __main____global_statements
__main__
[]
__main____global_statements
Public
)
})
main_program
[__main__]
[(SubroutineCall
6 __main____global_statements
2 __main____global_statements
[]
()
)]
)
})
[]
)
LPython naturally acts as a Python compiler. By default, if no backend is provided it compiles type-annotated user input code to LLVM, which generates binary final output. Consider the following small example:
from lpython import i32, i64
def list_bench(n: i32) -> i64:
x: list[i32]
x = []
i: i32
for i in range(n):
x.append(i)
s: i64 = i64(0)
for i in range(n):
s += i64(x[i])
return s
res: i64 = list_bench(500_000)
print(res)
(lp) 18:58:29:~/lpython_project/lpython % lpython /Users/czgdp1807/lpython_project/debug.py -o a.out
(lp) 18:58:31:~/lpython_project/lpython % time ./a.out
124999750000
./a.out 0.01s user 0.00s system 89% cpu 0.012 total
You can see that it’s very fast. It’s still plenty fast with the C backend via the command-line argument --backend=c:
% time lpython /Users/czgdp1807/lpython_project/debug.py --backend=c
124999750000
lpython /Users/czgdp1807/lpython_project/debug.py --backend=c 0.12s user 0.02s system 100% cpu 0.144 total
Note that time lpython /Users/czgdp1807/lpython_project/debug.py --backend=c includes both the compilation time of LPython and the execution time of the binary. The sum of both is so fast that one can afford to compile on every change to the input files. :D.
Just-in-time compilation in LPython requires only decorating Python function with @lpython. The decorator takes an option for specifying the desired backend, as in, @lpython(backend="c") or @lpython(backend="llvm"). Only C is supported at present; LLVM and others will be added in the near future. The decorator also propagates backend-specific options. For example
@lpython(backend="c",
backend_optimization_flags=["-ffast-math",
"-funroll-loops",
"-O1"])
Note that by default C backend is used without any optimisation flags.
A small example of JIT compilation in LPython (notice the LPython type annotations with the variables),
from lpython import i32, i64, lpython
@lpython(backend="c", backend_optimisation_flags=["-ffast-math", "-funroll-loops", "-O1"])
def list_bench(n: i32) -> i64:
x: list[i32]
x = []
i: i32
for i in range(n):
x.append(i)
s: i64 = i64(0)
for i in range(n):
s += i64(x[i])
return s
res = list_bench(1) # compiles `list_bench` to a shared binary in the first call
res = list_bench(500_000) # calls the compiled `list_bench`
print(res)
(lp) 18:46:33:~/lpython_project/lpython % python /Users/czgdp1807/lpython_project/debug.py
124999750000
We show below in the benchmarks how LPython compares to Numba, which also has JIT compilation.
Access any library implemented using CPython, via the @pythoncall decorator. For example,
email_extractor.py
# get_email is implemented in email_extractor_util.py which is intimated to
# LPython by specifiying the file as module in `@pythoncall` decorator
@pythoncall(module="email_extractor_util")
def get_email(text: str) -> str:
pass
def test():
text: str = "Hello, my email id is lpython@lcompilers.org."
print(get_email(text))
test()
email_extractor_util.py
# Implement `get_email` using `re` CPython library
def get_email(text):
import re
# Regular expression patterns
email_pattern = r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"
# Matching email addresses
email_matches = re.findall(email_pattern, text)
return email_matches[0]
(lp) 18:54:13:~/lpython_project % lpython email_extractor.py --backend=c --enable-cpython
lpython@lcompilers.org
Note: The @pythoncall and @lpython decorators are presently supported with just the C backend but eventually will work with the LLVM backend and that’s work in progress.
In this section, we show how LPython performs compares to competitors on each feature LPython offers. We cover JIT compilation, Interoperability with CPython, and AoT compilation.
We compare JIT compilation of LPython to Numba on summation of all the elements of a 1-D array, pointwise multiplication of two 1-D arrays, insertion sort on lists, and quadratic-time implementation of the Dijkstra shortest-path algorithm on a fully connected graph.
System Information
| Compiler | Version |
|---|---|
| Numba | 0.57.1 |
| LPython | 0.19.0 |
| Python | 3.10.4 |
Summation of all the elements of a 1-D array
from numpy import float64, arange, empty
from lpython import i32, f64, lpython
import timeit
from numba import njit
@lpython(backend="c", backend_optimisation_flags=["-ffast-math", "-funroll-loops", "-O3"])
def fast_sum(n: i32, x: f64[:], res: f64[:]) -> f64:
s: f64 = 0.0
res[0] = 0.0
i: i32
for i in range(n):
s += x[i]
res[0] = s
return s
@njit(fastmath=True)
def fast_sum_numba(n, x, res):
s = 0.0
res[0] = 0.0
for i in range(n):
s += x[i]
res[0] = s
return s
def test():
n = 100_000_000
x = arange(n, dtype=float64)
x1 = arange(0, dtype=float64)
res = empty(1, dtype=float64)
res_numba = empty(1, dtype=float64)
print("LPython compilation time:", timeit.timeit(lambda: fast_sum(0, x1, res), number=1))
print("LPython execution time: ", min(timeit.repeat(lambda: fast_sum(n, x, res), repeat=10, number=1)))
assert res[0] == 4999999950000000.0
print("Numba compilation time:", timeit.timeit(lambda: fast_sum_numba(0, x1, res_numba), number=1))
print("Numba execution time:", min(timeit.repeat(lambda: fast_sum_numba(n, x, res_numba), repeat=10, number=1)))
assert res_numba[0] == 4999999950000000.0
test()
| Compiler | Compilation Time (s) | System | Relative |
|---|---|---|---|
| Numba | 0.10 | Apple M1 MBP 2020 | 1.00 |
| LPython | 0.20 | Apple M1 MBP 2020 | 2.00 |
| Numba | 0.08 | Apple M1 Pro MBP 2021 | 1.00 |
| LPython | 0.53 | Apple M1 Pro MBP 2021 | 6.62 |
| Numba | 0.15 | Apple M1 2020 | 1.00 |
| LPython | 0.40 | Apple M1 2020 | 2.67 |
| Numba | 0.20 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| LPython | 0.32 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.60 |
| Compiler | Execution Time (s) | System | Relative |
|---|---|---|---|
| LPython | 0.013 | Apple M1 MBP 2020 | 1.00 |
| Numba | 0.024 | Apple M1 MBP 2020 | 1.84 |
| LPython | 0.013 | Apple M1 Pro MBP 2021 | 1.00 |
| Numba | 0.023 | Apple M1 Pro MBP 2021 | 1.77 |
| LPython | 0.014 | Apple M1 2020 | 1.00 |
| Numba | 0.024 | Apple M1 2020 | 1.71 |
| LPython | 0.048 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| Numba | 0.048 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
Pointwise multiplication of two 1-D arrays
from numpy import int64, arange, empty
from lpython import i32, i64, lpython
import timeit
from numba import njit
@lpython(backend="c", backend_optimisation_flags=["-ffast-math", "-funroll-loops", "-O3"])
def multiply_arrays(n: i32, x: i64[:], y: i64[:], z: i64[:]):
i: i32
for i in range(n):
z[i] = x[i] * y[i]
@njit(fastmath=True)
def multiply_arrays_numba(n, x, y, z):
for i in range(n):
z[i] = x[i] * y[i]
def test():
n = 100_000_000
x1 = arange(0, dtype=int64)
y1 = arange(0, dtype=int64)
res1 = arange(0, dtype=int64)
x = arange(n, dtype=int64)
y = arange(n, dtype=int64) + 2
res = empty(n, dtype=int64)
res_numba = empty(n, dtype=int64)
print("LPython compilation time:", timeit.timeit(lambda: multiply_arrays(0, x1, y1, res1), number=1))
print("LPython execution time:", min(timeit.repeat(lambda: multiply_arrays(n, x, y, res), repeat=10, number=1)))
assert sum(res - x * y) == 0
print("Numba compilation time:", timeit.timeit(lambda: multiply_arrays_numba(0, x1, y1, res1), number=1))
print("Numba execution time:", min(timeit.repeat(lambda: multiply_arrays_numba(n, x, y, res_numba), repeat=10, number=1)))
assert sum(res_numba - x * y) == 0
test()
| Compiler | Compilation Time (s) | System | Relative |
|---|---|---|---|
| Numba | 0.11 | Apple M1 MBP 2020 | 1.00 |
| LPython | 0.50 | Apple M1 MBP 2020 | 4.54 |
| Numba | 0.09 | Apple M1 Pro MBP 2021 | 1.00 |
| LPython | 0.60 | Apple M1 Pro MBP 2021 | 6.67 |
| Numba | 0.11 | Apple M1 2020 | 1.00 |
| LPython | 0.46 | Apple M1 2020 | 4.18 |
| Numba | 0.21 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| LPython | 0.31 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.48 |
| Compiler | Execution Time (s) | System | Relative |
|---|---|---|---|
| Numba | 0.041 | Apple M1 MBP 2020 | 1.00 |
| LPython | 0.042 | Apple M1 MBP 2020 | 1.02 |
| Numba | 0.037 | Apple M1 Pro MBP 2021 | 1.00 |
| LPython | 0.040 | Apple M1 Pro MBP 2021 | 1.08 |
| Numba | 0.042 | Apple M1 2020 | 1.00 |
| LPython | 0.042 | Apple M1 2020 | 1.00 |
| Numba | 0.21 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| LPython | 0.21 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
Insertion sort on lists
from lpython import i32, lpython
import timeit
from numba import njit
@lpython(backend="c", backend_optimisation_flags=["-ffast-math", "-funroll-loops", "-O3"])
def test_list_sort(size: i32):
i: i32
x: list[i32]
x = []
for i in range(size):
x.append(size - i)
for i in range(1, size):
key: i32 = x[i]
j: i32 = i - 1
while j >= 0 and key < x[j] :
x[j + 1] = x[j]
j -= 1
x[j + 1] = key
for i in range(1, size):
assert x[i - 1] < x[i]
@njit(fastmath=True)
def test_list_sort_numba(size):
x = []
for i in range(size):
x.append(size - i)
for i in range(1, size):
key = x[i]
j = i - 1
while j >= 0 and key < x[j] :
x[j + 1] = x[j]
j -= 1
x[j + 1] = key
for i in range(1, size):
assert x[i - 1] < x[i]
def test():
n = 25000
print("LPython compilation time:", timeit.timeit(lambda: test_list_sort(0), number=1))
print("LPython execution time:", min(timeit.repeat(lambda: test_list_sort(n), repeat=10, number=1)))
print("Numba compilation time:", timeit.timeit(lambda: test_list_sort_numba(0), number=1))
print("Numba execution time:", min(timeit.repeat(lambda: test_list_sort_numba(n), repeat=10, number=1)))
test()
| Compiler | Compilation Time (s) | System | Relative |
|---|---|---|---|
| Numba | 0.13 | Apple M1 MBP 2020 | 1.00 |
| LPython | 0.20 | Apple M1 MBP 2020 | 1.54 |
| Numba | 0.13 | Apple M1 Pro MBP 2021 | 1.00 |
| LPython | 0.60 | Apple M1 Pro MBP 2021 | 4.62 |
| Numba | 0.13 | Apple M1 2020 | 1.00 |
| LPython | 0.42 | Apple M1 2020 | 3.23 |
| Numba | 0.35 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| LPython | 0.37 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.06 |
| Compiler | Execution Time (s) | System | Relative |
|---|---|---|---|
| LPython | 0.11 | Apple M1 MBP 2020 | 1.00 |
| Numba | 0.39 | Apple M1 MBP 2020 | 3.54 |
| LPython | 0.11 | Apple M1 Pro MBP 2021 | 1.00 |
| Numba | 0.39 | Apple M1 Pro MBP 2021 | 3.54 |
| LPython | 0.20 | Apple M1 2020 | 1.00 |
| Numba | 0.39 | Apple M1 2020 | 1.95 |
| LPython | 0.10 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| Numba | 0.36 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 3.60 |
Quadratic-time implementation of the Dijkstra shortest-path algorithm on a fully connected graph
from lpython import i32, lpython
from numpy import empty, int32
from numba import njit
import timeit
@lpython(backend="c", backend_optimisation_flags=["-ffast-math", "-funroll-loops", "-O1"])
def dijkstra_shortest_path(n: i32, source: i32, dist_sum: i32[:]):
i: i32; j: i32; v: i32; u: i32; mindist: i32; alt: i32; dummy: i32;
graph: dict[i32, i32] = {}
dist: dict[i32, i32] = {}
prev: dict[i32, i32] = {}
visited: dict[i32, bool] = {}
Q: list[i32] = []
for i in range(n):
for j in range(n):
graph[n * i + j] = abs(i - j)
for v in range(n):
dist[v] = 2147483647
prev[v] = -1
Q.append(v)
visited[v] = False
dist[source] = 0
while len(Q) > 0:
u = -1
mindist = 2147483647
for i in range(len(Q)):
if mindist > dist[Q[i]]:
mindist = dist[Q[i]]
u = Q[i]
Q.remove(u)
visited[u] = True
for v in range(n):
if v != u and not visited[v]:
alt = dist[u] + graph[n * u + v]
if alt < dist[v]:
dist[v] = alt
prev[v] = u
dist_sum[0] = 0
for i in range(n):
dist_sum[0] += dist[i]
@njit(fastmath=True)
def dijkstra_shortest_path_numba(n, source, dist_sum):
graph = {}
dist = {}
prev = {}
visited = {}
Q = []
for i in range(n):
for j in range(n):
graph[n * i + j] = abs(i - j)
for v in range(n):
dist[v] = 2147483647
prev[v] = -1
Q.append(v)
visited[v] = False
dist[source] = 0
while len(Q) > 0:
u = -1
mindist = 2147483647
for i in range(len(Q)):
if mindist > dist[Q[i]]:
mindist = dist[Q[i]]
u = Q[i]
Q.remove(u)
visited[u] = True
for v in range(n):
if v != u and not visited[v]:
alt = dist[u] + graph[n * u + v]
if alt < dist[v]:
dist[v] = alt
prev[v] = u
dist_sum[0] = 0
for i in range(n):
dist_sum[0] += dist[i]
def test():
n: i32 = 4000
dist_sum_array_numba = empty(1, dtype=int32)
dist_sum_array = empty(1, dtype=int32)
print("LPython compilation time: ", timeit.timeit(lambda: dijkstra_shortest_path(0, 0, dist_sum_array), number=1))
print("LPython execution time: ", min(timeit.repeat(lambda: dijkstra_shortest_path(n, 0, dist_sum_array), repeat=5, number=1)))
print(dist_sum_array[0])
assert dist_sum_array[0] == i32(n * (n - 1)/2)
print("Numba compilation time: ", timeit.timeit(lambda: dijkstra_shortest_path_numba(0, 0, dist_sum_array_numba), number=1))
print("Numba execution time: ", min(timeit.repeat(lambda: dijkstra_shortest_path_numba(n, 0, dist_sum_array_numba), repeat=5, number=1)))
print(dist_sum_array_numba[0])
assert dist_sum_array_numba[0] == i32(n * (n - 1)/2)
test()
| Compiler | Compilation Time (s) | System | Relative |
|---|---|---|---|
| LPython | 0.35 | Apple M1 MBP 2020 | 1.00 |
| Numba | 0.81 | Apple M1 MBP 2020 | 2.31 |
| LPython | 0.69 | Apple M1 Pro MBP 2021 | 1.00 |
| Numba | 0.73 | Apple M1 Pro MBP 2021 | 1.05 |
| LPython | 0.21 | Apple M1 2020 | 1.00 |
| Numba | 0.73 | Apple M1 2020 | 3.47 |
| LPython | 1.08 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| Numba | 1.69 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.56 |
| Compiler | Execution Time (s) | System | Relative |
|---|---|---|---|
| LPython | 0.23 | Apple M1 MBP 2020 | 1.00 |
| Numba | 1.01 | Apple M1 MBP 2020 | 4.39 |
| LPython | 0.20 | Apple M1 Pro MBP 2021 | 1.00 |
| Numba | 0.98 | Apple M1 Pro MBP 2021 | 4.90 |
| LPython | 0.27 | Apple M1 2020 | 1.00 |
| Numba | 0.98 | Apple M1 2020 | 3.63 |
| LPython | 0.87 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| Numba | 1.95 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 2.24 |
Next, we see how LPython compares to other AoT compilers and to the standard CPython interpreter. The tasks considered are quadratic-time implementation of the Dijkstra shortest-path algorithm on a fully connected graph, and Floyd-Warshall algorithm on array representation of graphs.
System Information
| Compiler | Version |
|---|---|
| clang++ | 14.0.3 |
| g++ | 11.3.0 |
| LPython | 0.19.0 |
| Python | 3.10.4 |
Quadratic-time implementation of the Dijkstra shortest-path algorithm on a fully connected graph
from lpython import i32
def dijkstra_shortest_path(n: i32, source: i32) -> i32:
i: i32; j: i32; v: i32; u: i32; mindist: i32; alt: i32; dummy: i32; uidx: i32
dist_sum: i32;
graph: dict[i32, i32] = {}
dist: dict[i32, i32] = {}
prev: dict[i32, i32] = {}
visited: dict[i32, bool] = {}
Q: list[i32] = []
for i in range(n):
for j in range(n):
graph[n * i + j] = abs(i - j)
for v in range(n):
dist[v] = 2147483647
prev[v] = -1
Q.append(v)
visited[v] = False
dist[source] = 0
while len(Q) > 0:
u = -1
mindist = 2147483647
for i in range(len(Q)):
if mindist > dist[Q[i]]:
mindist = dist[Q[i]]
u = Q[i]
uidx = i
dummy = Q.pop(uidx)
visited[u] = True
for v in range(n):
if v != u and not visited[v]:
alt = dist[u] + graph[n * u + v]
if alt < dist[v]:
dist[v] = alt
prev[v] = u
dist_sum = 0
for i in range(n):
dist_sum += dist[i]
return dist_sum
def test():
n: i32 = 4000
print(dijkstra_shortest_path(n, 0))
test()
#include <iostream>
#include <unordered_map>
#include <vector>
int32_t dijkstra_shortest_path(int32_t n, int32_t source) {
int32_t i, j, v, u, mindist, alt, dummy, uidx;
std::unordered_map<int32_t, int32_t> graph, dist, prev;
std::unordered_map<int32_t, bool> visited;
std::vector<int32_t> Q;
for(i = 0; i < n; i++) {
for(j = 0; j < n; j++) {
graph[n * i + j] = std::abs(i - j);
}
}
for(v = 0; v < n; v++) {
dist[v] = 2147483647;
prev[v] = -1;
Q.push_back(v);
visited[v] = false;
}
dist[source] = 0;
while(Q.size() > 0) {
u = -1;
mindist = 2147483647;
for(i = 0; i < Q.size(); i++) {
if( mindist > dist[Q[i]] ) {
mindist = dist[Q[i]];
u = Q[i];
uidx = i;
}
}
Q.erase(Q.begin() + uidx);
visited[u] = true;
for(v = 0; v < n; v++) {
if( v != u and not visited[v] ) {
alt = dist[u] + graph[n * u + v];
if( alt < dist[v] ) {
dist[v] = alt;
prev[v] = u;
}
}
}
}
int32_t dist_sum = 0;
for(i = 0; i < n; i++) {
dist_sum += dist[i];
}
return dist_sum;
}
int main() {
int32_t n = 4000;
int32_t dist_sum = dijkstra_shortest_path(n, 0);
std::cout<<dist_sum<<std::endl;
return 0;
}
| Compiler/Interpreter | Execution Time (s) | System | Relative |
|---|---|---|---|
| LPython | 0.167 | Apple M1 MBP 2020 | 1.00 |
| Clang++ | 0.993 | Apple M1 MBP 2020 | 5.95 |
| Python | 3.817 | Apple M1 MBP 2020 | 22.86 |
| LPython | 0.155 | Apple M1 Pro MBP 2021 | 1.00 |
| Clang++ | 0.685 | Apple M1 Pro MBP 2021 | 4.41 |
| Python | 3.437 | Apple M1 Pro MBP 2021 | 22.17 |
| LPython | 0.324 | Apple M1 2020 | 1.00 |
| Clang++ | 0.709 | Apple M1 2020 | 2.19 |
| Python | 3.486 | Apple M1 2020 | 10.76 |
| LPython | 0.613 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| g++ | 1.358 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 2.21 |
| Python | 7.365 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 12.01 |
Note the optimization flags furnished to each compiler.
| Compiler/Interpreter | Optimization flags used |
|---|---|
| LPython | --fast |
| Clang++ | -ffast-math -funroll-loops -O3 |
| g++ | -ffast-math -funroll-loops -O3 |
| Python | - |
Floyd-Warshall algorithm on array representation of graphs
from lpython import i64, i32
from numpy import empty, int64
def floyd_warshall(size: i32) -> i64:
dist: i64[size, size] = empty((size, size), dtype=int64)
u: i32; v: i32
i: i32; j: i32; k: i32
update: i64 = i64(0)
for u in range(size):
for v in range(size):
dist[u, v] = i64(2147483647)
for u in range(size):
for v in range(size):
if u != v and ((u%2 == 0 and v%2 == 1)
or (u%2 == 1 and v%2 == 0)):
dist[u, v] = i64(u + v)
for v in range(size):
dist[v, v] = i64(0)
update = i64(0)
for k in range(size):
for i in range(size):
for j in range(size):
if dist[i, j] > dist[i, k] + dist[k, j]:
update += dist[i, j] - dist[i, k] - dist[k, j]
dist[i, j] = dist[i, k] + dist[k, j]
return update
print(floyd_warshall(1000))
#include <iostream>
int64_t floyd_warshall(int32_t size) {
int64_t dist[size][size];
int32_t u, v, i, j, k;
int64_t update;
for(u = 0; u < size; u++) {
for(v = 0; v < size; v++) {
dist[u][v] = 2147483647;
}
}
for(u = 0; u < size; u++) {
for(v = 0; v < size; v++) {
if( u != v && ((u%2 == 0 and v%2 == 1)
|| (u%2 == 1 and v%2 == 0)) ) {
dist[u][v] = u + v;
}
}
}
for(v = 0; v < size; v++) {
dist[v][v] = 0;
}
update = 0;
for(k = 0; k < size; k++) {
for(i = 0; i < size; i++) {
for(j = 0; j < size; j++) {
if( dist[i][j] > dist[i][k] + dist[k][j] ) {
update += dist[i][j] - dist[i][k] - dist[k][j];
dist[i][j] = dist[i][k] + dist[k][j];
}
}
}
}
return update;
}
int main() {
std::cout<<(floyd_warshall(1000))<<std::endl;
return 0;
}
| Compiler/Interpreter | Execution Time (s) | System | Relative |
|---|---|---|---|
| Clang++ | 0.451 | Apple M1 MBP 2020 | 1.00 |
| LPython | 0.767 | Apple M1 MBP 2020 | 1.70 |
| Python | > 11 | Apple M1 MBP 2020 | > 24.39 |
| Clang++ | 0.435 | Apple M1 Pro MBP 2021 | 1.00 |
| LPython | 0.785 | Apple M1 Pro MBP 2021 | 1.80 |
| Python | > 11 | Apple M1 Pro MBP 2021 | > 25.28 |
| Clang++ | 0.460 | Apple M1 2020 | 1.00 |
| LPython | 0.995 | Apple M1 2020 | 2.16 |
| Python | > 11 | Apple M1 2020 | > 23.91 |
| g++ | 0.695 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 1.00 |
| LPython | 2.933 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 4.22 |
| Python | 440.588 | AMD Ryzen 5 2500U (Ubuntu 22.04) | 633.94 |
Note the optimization flags furnished to each compiler.
| Compiler/Interpreter | Optimization flags used |
|---|---|
| LPython | --fast |
| Clang++ | -ffast-math -funroll-loops -O3 |
| g++ | -ffast-math -funroll-loops -O3 |
| Python | - |
Next we show that LPython can call functions in CPython libraries. This feature permits “break-out” to Numpy, TensorFlow, PyTorch, and even to matplotlib. The break-outs will run at ordinary (slow) Python speeds, but LPython accelerates the mathematical portions to near maximum speed.
Calling NumPy functions via CPython
main.py
from lpython import i32, f64, i64, pythoncall, Const, TypeVar
from numpy import empty, int32, float64
n_1 = TypeVar("n_1")
n_2 = TypeVar("n_2")
n_3 = TypeVar("n_3")
@pythoncall(module = "util")
def cpython_add(n_1: i32, a: i32[:], b: i32[:]) -> i32[n_1]:
pass
@pythoncall(module = "util")
def cpython_multiply(n_1: i32, n_2: i32, a: f64[:], b: f64[:]) -> f64[n_1, n_2]:
pass
def test_1D():
n: Const[i32] = 500_000
a: i32[n] = empty(n, dtype = int32)
b: i32[n] = empty(n, dtype = int32)
i: i32
for i in range(n):
a[i] = 2 * (i+1) * 13
b[i] = a[i] + 2
sum: i32[n]
sum = cpython_add(500_000, a, b)
for i in range(n):
assert sum[i] == a[i] + b[i]
def test_2D():
n: Const[i32] = 1_000
a: f64[n] = empty([n], dtype = float64)
b: f64[n] = empty([n], dtype = float64)
i: i32; j: i32
for i in range(n):
a[i] = f64(i + 13)
b[i] = i * 2 / (i + 1)
product: f64[n, n]
product = cpython_multiply(1_000, 1_000, a, b)
for i in range(n):
assert product[i] == a[i] * b[i]
def test():
test_1D()
test_2D()
test()
util.py
import numpy as np
def cpython_add(n, a, b):
return np.add(a, b)
def cpython_multiply(n, m, a, b):
return np.multiply(a, b)
(lp) 23:02:55:~/lpython_project % lpython main.py --backend=c --link-numpy
(lp) 23:03:10:~/lpython_project % # Works successfully without any asserts failing
Plotting graphs via Matplotlib
main.py
from lpython import f64, i32, pythoncall, Const
from numpy import empty, float64
@pythoncall(module = "util")
def plot_graph(x: f64[:], y1: f64[:], y2: f64[:], y3: f64[:]):
pass
def f(x: f64, i: f64) -> f64:
return x ** .5 / i
def test():
n: Const[i32] = 100000
x: f64[n] = empty(n, dtype=float64)
y1: f64[n] = empty(n, dtype=float64)
y2: f64[n] = empty(n, dtype=float64)
y3: f64[n] = empty(n, dtype=float64)
i: i32
for i in range(1, n):
x[i] = f64(i)
for i in range(1, n):
y1[i] = f(x[i], 1.)
y2[i] = f(x[i], 2.)
y3[i] = f(x[i], 3.)
plot_graph(x, y1, y2, y3)
test()
util.py
import matplotlib.pyplot as plt
def plot_graph(x, y1, y2, y3):
plt.figtext(0.92, 0.03, '$x$')
plt.figtext(0.1, 0.9, '$y$')
plt.plot(x, y1, label="y1")
plt.plot(x, y2, label="y2")
plt.plot(x, y3, label="y3")
plt.legend()
plt.savefig('graph.png')
plt.show()
(lp) 23:09:08:~/lpython_project % lpython main.py --backend=c --link-numpy
(lp) 23:10:44:~/lpython_project % # Works see the graph below
Visualization using Matplotlib: Mandelbrot Set
main.py
from lpython import i32, f64, pythoncall, TypeVar
from numpy import empty, int32
h = TypeVar("h")
w = TypeVar("w")
d = TypeVar("d")
@pythoncall(module="util")
def show_img_gray(w: i32, h: i32, A: i32[h, w]):
pass
@pythoncall(module="util")
def show_img_color(w: i32, h: i32, d: i32, A: i32[h, w, d]):
pass
def main0():
Nx: i32 = 600; Ny: i32 = 450; Nz: i32 = 4; n_max: i32 = 255
xcenter: f64 = f64(-0.5); ycenter: f64 = f64(0.0)
width: f64 = f64(4); height: f64 = f64(3)
dx_di: f64 = width/f64(Nx); dy_dj: f64 = -height/f64(Ny)
x_offset: f64 = xcenter - f64(Nx+1)*dx_di/f64(2.0)
y_offset: f64 = ycenter - f64(Ny+1)*dy_dj/f64(2.0)
i: i32; j: i32; n: i32; idx: i32
x: f64; y: f64; x_0: f64; y_0: f64; x_sqr: f64; y_sqr: f64
image: i32[450, 600] = empty([Ny, Nx], dtype=int32)
image_color: i32[450, 600, 4] = empty([Ny, Nx, Nz], dtype=int32)
palette: i32[4, 3] = empty([4, 3], dtype=int32)
for j in range(Ny):
y_0 = y_offset + dy_dj * f64(j + 1)
for i in range(Nx):
x_0 = x_offset + dx_di * f64(i + 1)
x = 0.0; y = 0.0; n = 0
while(True):
x_sqr = x ** 2.0
y_sqr = y ** 2.0
if (x_sqr + y_sqr > f64(4) or n == n_max):
image[j,i] = 255 - n
break
y = y_0 + f64(2.0) * x * y
x = x_0 + x_sqr - y_sqr
n = n + 1
palette[0,0] = 0; palette[0,1] = 135; palette[0,2] = 68
palette[1,0] = 0; palette[1,1] = 87; palette[1,2] = 231
palette[2,0] = 214; palette[2,1] = 45; palette[2,2] = 32
palette[3,0] = 255; palette[3,1] = 167; palette[3,2] = 0
for j in range(Ny):
for i in range(Nx):
idx = image[j,i] - i32(image[j,i]/4)*4
image_color[j,i,0] = palette[idx,0] # Red
image_color[j,i,1] = palette[idx,1] # Green
image_color[j,i,2] = palette[idx,2] # Blue
image_color[j,i,3] = 255 # Alpha
show_img_gray(Nx, Ny, image)
show_img_color(Nx, Ny, Nz, image_color)
print("Done.")
main0()
util.py
def show_img_gray(w, h, A):
from matplotlib import pyplot as plt
plt.imshow(A, cmap='gray')
plt.show()
plt.close()
def show_img_color(w, h, d, A):
from matplotlib import pyplot as plt
plt.imshow(A)
plt.show()
plt.close()
$ ls
main.py util.py
$ lpython main.py --backend=c --link-numpy
Done.
The benchmarks support the claim that LPython is competitive with its competitors in all features it offers. In JIT, the execution times of LPython-compiled functions are at least as short as equivalent Numba functions. The speed of JIT compilation, itself, is slow in some cases because it currently depends on a C compiler to generate optimal binary code. For algorithms with rich data structures like dict (hash maps) and list, LPython shows much faster speed than Numba. In AoT compilation for tasks like the Dijkstra algorithm, LPython beats equivalent C++ code very comfortably. For an array-based implementation of the Floyd-Warshall algorithm, LPython generates code almost as fast as C++ does.
The main takeaway is that LPython/LFortran generate fast code by default. Our benchmarks show that it’s straightforward to write high-speed LPython code. We hope to raise expectations that LPython output will be in general at least as fast as the equivalent C++ code. Users love Python because of its many productivity advantages: great tooling, easy syntax, and rich data structures like lists, dicts, sets, and arrays. Because any LPython program is also an ordinary Python program, all the tools – debuggers and profilers, for instance – just work. Then, LPython delivers run-time speeds, even with rich data structures at least as short as alternatives in most cases.