Compare commits
41 Commits
preview
...
vec-based-
| Author | SHA1 | Date | |
|---|---|---|---|
| 21228ff3d7 | |||
| 588b3b5b2c | |||
| f6152670aa | |||
| c2b9ee71b8 | |||
| f8e5bd7423 | |||
| d7001a5c52 | |||
| bc68d9fa49 | |||
| 264d8f92f4 | |||
| d8f5b876ac | |||
| 8cf6177cbc | |||
| 39bd4400b4 | |||
| 75b99869d4 | |||
| de0bbb8171 | |||
| 92f59cbf9a | |||
| dd9ca660cc | |||
| 7e2ef49481 | |||
| 86130984e2 | |||
| c4b146c325 | |||
| 7b6fc89fb7 | |||
| 8c9756b6d2 | |||
| 02993142df | |||
| 3348b7cf6d | |||
| 3098dc7e0a | |||
| e0c00019ff | |||
| 35fbae8ab9 | |||
| 23d336d63e | |||
| 39351e1131 | |||
| b7872da3ea | |||
| 5cc89b855a | |||
| 32e4f1ea4f | |||
| b664297c73 | |||
| ea60f17647 | |||
| 5ffa0ea2ec | |||
| 2a59fe8c84 | |||
| 8f79440219 | |||
| 128b05b8a8 | |||
| a9ee8eb66c | |||
| 5c7b6a7b41 | |||
| a569781691 | |||
| 0b75c30784 | |||
| ed2ae144dd |
@@ -4,3 +4,5 @@ version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
[dependencies]
|
||||
anyhow = "1.0.53"
|
||||
thiserror = "1.0.30"
|
||||
|
||||
207
README.md
207
README.md
@@ -1,38 +1,195 @@
|
||||
# NEK-Lang
|
||||
|
||||
## Variables
|
||||
Currently all variables are global and completely unscoped. That means no matter where a variable is declared, it remains over the whole remaining runtime of the progam.
|
||||
|
||||
All variables are currently of type `i64` (64-bit signed integer)
|
||||
|
||||
### Declaration
|
||||
- Declare and initialize a new variable
|
||||
- Declaring a previously declared variable again is currently equivalent to an assignment
|
||||
- Declaration is needed before assignment or other usage
|
||||
- The variable name is on the left side of the `<-` operator
|
||||
- The assigned value is on the right side and can be any expression
|
||||
```
|
||||
a <- 123;
|
||||
```
|
||||
Create a new variable named `a` and assign the value `123` to it.
|
||||
|
||||
### Assignment
|
||||
- Assigning a value to a previously declared variable
|
||||
- The variable name is on the left side of the `=` operator
|
||||
- The assigned value is on the right side and can be any expression
|
||||
```
|
||||
a = 123;
|
||||
```
|
||||
The value `123` is assigned to the variable named `a`. `a` needs to be declared before this.
|
||||
|
||||
## Expressions
|
||||
The operator precedence is the same order as in `C` for all implemented operators.
|
||||
Refer to the
|
||||
[C Operator Precedence Table](https://en.cppreference.com/w/c/language/operator_precedence)
|
||||
to see the different precedences.
|
||||
|
||||
### General
|
||||
- Parentheses `(` and `)` can be used to modify evaluation oder just like in any other
|
||||
programming language.
|
||||
- For example `(a + b) * c` will evaluate the addition before the multiplication, despite the multiplication having higher binding power
|
||||
|
||||
### Mathematical Operators
|
||||
Supported mathematical operations:
|
||||
- Addition `a + b`
|
||||
- Subtraction `a - b`
|
||||
- Multiplication `a * b`
|
||||
- Division `a / b`
|
||||
- Modulo `a % b`
|
||||
- Negation `-a`
|
||||
|
||||
### Bitwise Operators
|
||||
- And `a & b`
|
||||
- Or `a | b`
|
||||
- Xor `a ^ b`
|
||||
- Bitshift left (by `b` bits) `a << b`
|
||||
- Bitshift right (by `b` bits) `a >> b`
|
||||
- "Bit flip" (One's complement) `~a`
|
||||
|
||||
### Logical Operators
|
||||
The logical operators evaluate the operands as `false` if they are equal to `0` and `true` if they are not equal to `0`
|
||||
- And `a && b`
|
||||
- Or `a || b`
|
||||
- Not `!a` (if `a` is equal to `0`, the result is `1`, otherwise the result is `0`)
|
||||
|
||||
### Equality & Relational Operators
|
||||
The equality and relational operations result in `1` if the condition is evaluated as `true` and in `0` if the condition is evaluated as `false`.
|
||||
- Equality `a == b`
|
||||
- Inequality `a != b`
|
||||
- Greater than `a > b`
|
||||
- Greater or equal than `a >= b`
|
||||
- Less than `a < b`
|
||||
- Less or equal than `a <= b`
|
||||
|
||||
## Control-Flow
|
||||
For conditions like in if or loops, every non zero value is equal to `true`, and `0` is `false`.
|
||||
|
||||
### Loop
|
||||
- There is currently only the `loop` keyword that can act like a `while` with optional advancement (an expression that is executed after the loop body)
|
||||
- The `loop` keyword is followed by the condition (an expression) without needing parentheses
|
||||
- *Optional:* If there is a `;` after the condition, there must be another expression which is used as the advancement
|
||||
- The loops body is wrapped in braces (`{ }`) just like in C/C++
|
||||
|
||||
```
|
||||
// Print the numbers from 0 to 9
|
||||
|
||||
// Without advancement
|
||||
i <- 0;
|
||||
loop i < 10 {
|
||||
print i;
|
||||
i = i - 1;
|
||||
}
|
||||
|
||||
// With advancement
|
||||
k <- 0;
|
||||
loop k < 10; k = k - 1 {
|
||||
print k;
|
||||
}
|
||||
```
|
||||
|
||||
### If / Else
|
||||
|
||||
- The language supports `if` and an optional `else`
|
||||
- After the `if` keyword must be the deciding condition, parentheses are not needed
|
||||
- The block *if-true* block is wrapped in braces (`{ }`)
|
||||
- *Optional:* If there is an `else` after the *if-block*, there must be a following *if-false*, aka. else block
|
||||
```
|
||||
a <- 1;
|
||||
b <- 2;
|
||||
if a == b {
|
||||
// a is equal to b
|
||||
print 1;
|
||||
} else {
|
||||
// a is not equal to b
|
||||
print 0;
|
||||
}
|
||||
```
|
||||
|
||||
## IO
|
||||
|
||||
### Print
|
||||
Printing is implemented via the `print` keyword
|
||||
- The `print` keyword is followed by an expression, the value of which will be printed to the terminal.
|
||||
- Print currently automatically adds a linebreak
|
||||
```
|
||||
a <- 1;
|
||||
print a; // Outputs `"1\n"` to the terminal
|
||||
```
|
||||
|
||||
## Comments
|
||||
|
||||
### Line comments
|
||||
Line comments can be initiated by using `//`
|
||||
- Everything after `//` up to the end of the current line is ignored and not parsed
|
||||
```
|
||||
// This is a comment
|
||||
```
|
||||
|
||||
|
||||
# Feature Tracker
|
||||
|
||||
## High level Components
|
||||
|
||||
- [x] Lexer: Transforms text into Tokens
|
||||
- [x] Parser: Transforms Tokens into Abstract Syntax Tree
|
||||
- [x] Interpreter (tree-walk-interpreter): Walks the tree and evaluates the expressions / statements
|
||||
- [ ] Abstract Syntax Tree Optimizer
|
||||
|
||||
## Language features
|
||||
|
||||
- [x] Math expressions
|
||||
- [x] Unary operators
|
||||
- [x] Negate `-X`
|
||||
- [x] Parentheses `(X+Y)*Z`
|
||||
- [x] General expressions
|
||||
- [x] Arithmetic operations
|
||||
- [x] Addition `a + b`
|
||||
- [x] Subtraction `a - b`
|
||||
- [x] Multiplication `a * b`
|
||||
- [x] Division `a / b`
|
||||
- [x] Modulo `a % b
|
||||
- [x] Negate `-a`
|
||||
- [x] Parentheses `(a + b) * c`
|
||||
- [x] Logical boolean operators
|
||||
- [x] Equal `a == b`
|
||||
- [x] Not equal `a != b`
|
||||
- [x] Greater than `a > b`
|
||||
- [x] Less than `a < b`
|
||||
- [x] Greater than or equal `a >= b`
|
||||
- [x] Less than or equal `a <= b`
|
||||
- [x] Logical operators
|
||||
- [x] And `a && b`
|
||||
- [x] Or `a || b`
|
||||
- [x] Not `!a`
|
||||
- [x] Bitwise operators
|
||||
- [x] Bitwise AND `a & b`
|
||||
- [x] Bitwise OR `a | b`
|
||||
- [x] Bitwise XOR `a ^ b`
|
||||
- [x] Bitwise NOT `~a`
|
||||
- [x] Bitwise left shift `a << b`
|
||||
- [x] Bitwise right shift `a >> b`
|
||||
- [x] Variables
|
||||
- [x] Declaration
|
||||
- [x] Assignment
|
||||
- [x] While loop `while X { ... }`
|
||||
- [x] If else statement `if X { ... } else { ... }`
|
||||
- [x] If Statement
|
||||
- [x] Else statement
|
||||
- [ ] Line comments `//`
|
||||
- [x] Statements with semicolon & Multiline programs
|
||||
- [x] Control flow
|
||||
- [x] While loop `while X { ... }`
|
||||
- [x] If else statement `if X { ... } else { ... }`
|
||||
- [x] If Statement
|
||||
- [x] Else statement
|
||||
- [x] Line comments `//`
|
||||
- [x] Strings
|
||||
- [x] For loops `for X; Y; Z { ... }`
|
||||
- [ ] IO Intrinsics
|
||||
- [x] IO Intrinsics
|
||||
- [x] Print
|
||||
- [ ] ReadLine
|
||||
|
||||
## Grammar
|
||||
|
||||
### Expressions
|
||||
```
|
||||
LITERAL = I64 | Str
|
||||
expr_primary = LITERAL | IDENT | "(" expr ")" | "-" expr_primary
|
||||
LITERAL = I64_LITERAL | STR_LITERAL
|
||||
expr_primary = LITERAL | IDENT | "(" expr ")" | "-" expr_primary | "~" expr_primary
|
||||
expr_mul = expr_primary (("*" | "/" | "%") expr_primary)*
|
||||
expr_add = expr_mul (("+" | "-") expr_mul)*
|
||||
expr_shift = expr_add ((">>" | "<<") expr_add)*
|
||||
@@ -41,17 +198,15 @@ expr_equ = expr_rel (("==" | "!=") expr_rel)*
|
||||
expr_band = expr_equ ("&" expr_equ)*
|
||||
expr_bxor = expr_band ("^" expr_band)*
|
||||
expr_bor = expr_bxor ("|" expr_bxor)*
|
||||
expr = expr_bor
|
||||
expr_land = expr_bor ("&&" expr_bor)*
|
||||
expr_lor = expr_land ("||" expr_land)*
|
||||
expr = expr_lor
|
||||
```
|
||||
|
||||
## Statements
|
||||
```
|
||||
stmt_expr = expr
|
||||
stmt_let = "let" IDENT "=" expr
|
||||
stmt_while = "while" expr "{" (stmt)* "}"
|
||||
stmt_for = "for" stmt_let ";" expr ";" expr "{" (stmt)* "}"
|
||||
stmt_if = "if" expr "{" (stmt)* "}" ( "else" "{" (stmt)* "}" )
|
||||
stmt_dbgprint = "$$" expr
|
||||
stmt_print = "$" expr
|
||||
stmt = stmt_expr | stmt_let | stmt_while | stmt_for | stmt_if | stmt_dbgprint | stmt_print
|
||||
### Statements
|
||||
```
|
||||
stmt_if = "if" expr "{" stmt* "}" ("else" "{" stmt* "}")?
|
||||
stmt_loop = "loop" expr (";" expr)? "{" stmt* "}"
|
||||
stmt_expr = expr ";"
|
||||
stmt = stmt_expr | stmt_loop
|
||||
```
|
||||
15
examples/euler1.nek
Normal file
15
examples/euler1.nek
Normal file
@@ -0,0 +1,15 @@
|
||||
// If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9.
|
||||
// The sum of these multiples is 23.
|
||||
// Find the sum of all the multiples of 3 or 5 below 1000.
|
||||
//
|
||||
// Correct Answer: 233168
|
||||
|
||||
sum <- 0;
|
||||
i <- 0;
|
||||
loop i < 1_000; i = i + 1 {
|
||||
if i % 3 == 0 | i % 5 == 0 {
|
||||
sum = sum + i;
|
||||
}
|
||||
}
|
||||
|
||||
print sum;
|
||||
26
examples/euler2.nek
Normal file
26
examples/euler2.nek
Normal file
@@ -0,0 +1,26 @@
|
||||
// Each new term in the Fibonacci sequence is generated by adding the previous two terms.
|
||||
// By starting with 1 and 2, the first 10 terms will be:
|
||||
// 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...
|
||||
// By considering the terms in the Fibonacci sequence whose values do not exceed four million,
|
||||
// find the sum of the even-valued terms.
|
||||
//
|
||||
// Correct Answer: 4613732
|
||||
|
||||
sum <- 0;
|
||||
|
||||
a <- 0;
|
||||
b <- 1;
|
||||
tmp <- 0;
|
||||
|
||||
loop a < 4_000_000 {
|
||||
if a % 2 == 0 {
|
||||
sum = sum + a;
|
||||
}
|
||||
|
||||
tmp = a;
|
||||
a = b;
|
||||
b = b + tmp;
|
||||
}
|
||||
|
||||
print sum;
|
||||
|
||||
29
examples/euler3.nek
Normal file
29
examples/euler3.nek
Normal file
@@ -0,0 +1,29 @@
|
||||
// The prime factors of 13195 are 5, 7, 13 and 29.
|
||||
// What is the largest prime factor of the number 600851475143 ?
|
||||
//
|
||||
// Correct Answer: 6857
|
||||
|
||||
number <- 600_851_475_143;
|
||||
result <- 0;
|
||||
|
||||
div <- 2;
|
||||
|
||||
loop number > 1 {
|
||||
loop number % div == 0 {
|
||||
if div > result {
|
||||
result = div;
|
||||
}
|
||||
number = number / div;
|
||||
}
|
||||
|
||||
div = div + 1;
|
||||
if div * div > number {
|
||||
if number > 1 & number > result {
|
||||
result = number;
|
||||
}
|
||||
number = 0;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
print result;
|
||||
36
examples/euler4.nek
Normal file
36
examples/euler4.nek
Normal file
@@ -0,0 +1,36 @@
|
||||
// A palindromic number reads the same both ways. The largest palindrome made from the product of
|
||||
// two 2-digit numbers is 9009 = 91 × 99.
|
||||
// Find the largest palindrome made from the product of two 3-digit numbers.
|
||||
//
|
||||
// Correct Answer: 906609
|
||||
|
||||
|
||||
res <- 0;
|
||||
|
||||
tmp <- 0;
|
||||
num <- 0;
|
||||
num_rev <- 0;
|
||||
|
||||
i <- 100;
|
||||
k <- 100;
|
||||
loop i < 1_000; i = i + 1 {
|
||||
k = 100;
|
||||
loop k < 1_000; k = k + 1 {
|
||||
num_rev = 0;
|
||||
|
||||
num = i * k;
|
||||
|
||||
tmp = num;
|
||||
|
||||
loop tmp {
|
||||
num_rev = num_rev*10 + tmp % 10;
|
||||
tmp = tmp / 10;
|
||||
}
|
||||
|
||||
if num == num_rev & num > res {
|
||||
res = num;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
print res;
|
||||
24
examples/euler4.py
Normal file
24
examples/euler4.py
Normal file
@@ -0,0 +1,24 @@
|
||||
# A palindromic number reads the same both ways. The largest palindrome made from the product of
|
||||
# two 2-digit numbers is 9009 = 91 × 99.
|
||||
# Find the largest palindrome made from the product of two 3-digit numbers.
|
||||
#
|
||||
# Correct Answer: 906609
|
||||
|
||||
|
||||
res = 0
|
||||
|
||||
for i in range(100, 999):
|
||||
for k in range(100, 999):
|
||||
|
||||
num = i * k
|
||||
tmp = num
|
||||
|
||||
num_rev = 0
|
||||
while tmp != 0:
|
||||
num_rev = num_rev*10 + tmp % 10
|
||||
tmp = tmp // 10
|
||||
|
||||
if num == num_rev and num > res:
|
||||
res = num
|
||||
|
||||
print(res)
|
||||
28
examples/euler5.nek
Normal file
28
examples/euler5.nek
Normal file
@@ -0,0 +1,28 @@
|
||||
// 2520 is the smallest number that can be divided by each of the numbers from 1 to 10 without any remainder.
|
||||
// What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20?
|
||||
//
|
||||
// Correct Answer: 232_792_560
|
||||
|
||||
num <- 20;
|
||||
should_continue <- 1;
|
||||
i <- 2;
|
||||
|
||||
loop should_continue {
|
||||
should_continue = 0;
|
||||
|
||||
i = 20;
|
||||
loop i >= 2; i = i - 1 {
|
||||
if num % i != 0 {
|
||||
should_continue = 1;
|
||||
|
||||
// break
|
||||
i = 0;
|
||||
}
|
||||
}
|
||||
|
||||
if should_continue == 1 {
|
||||
num = num + 20;
|
||||
}
|
||||
}
|
||||
|
||||
print num;
|
||||
158
src/ast.rs
158
src/ast.rs
@@ -18,6 +18,24 @@ pub enum BinOpType {
|
||||
/// Modulo
|
||||
Mod,
|
||||
|
||||
/// Compare Equal
|
||||
EquEqu,
|
||||
|
||||
/// Compare Not Equal
|
||||
NotEqu,
|
||||
|
||||
/// Less than
|
||||
Less,
|
||||
|
||||
/// Less than or Equal
|
||||
LessEqu,
|
||||
|
||||
/// Greater than
|
||||
Greater,
|
||||
|
||||
/// Greater than or Equal
|
||||
GreaterEqu,
|
||||
|
||||
/// Bitwise OR (inclusive or)
|
||||
BOr,
|
||||
|
||||
@@ -27,78 +45,106 @@ pub enum BinOpType {
|
||||
/// Bitwise Xor (exclusive or)
|
||||
BXor,
|
||||
|
||||
/// Logical And
|
||||
LAnd,
|
||||
|
||||
/// Logical Or
|
||||
LOr,
|
||||
|
||||
/// Shift Left
|
||||
Shl,
|
||||
|
||||
/// Shift Right
|
||||
Shr,
|
||||
|
||||
/// Check equality
|
||||
Equ,
|
||||
|
||||
/// Check unequality
|
||||
Neq,
|
||||
|
||||
/// Check greater than
|
||||
Gt,
|
||||
|
||||
/// Check greater or equal
|
||||
Ge,
|
||||
|
||||
/// Check less than
|
||||
Lt,
|
||||
|
||||
/// Check less or equal
|
||||
Le,
|
||||
|
||||
/// Assign to a variable
|
||||
/// Assign value to variable
|
||||
Assign,
|
||||
|
||||
/// Declare new variable with value
|
||||
Declare,
|
||||
}
|
||||
|
||||
/// Types for unary operators
|
||||
#[derive(Debug, PartialEq, Eq, Clone)]
|
||||
pub enum UnOpType {
|
||||
/// Negation
|
||||
Neg,
|
||||
}
|
||||
/// Unary Negate
|
||||
Negate,
|
||||
|
||||
/// A full program abstract syntax tree. This consists of zero or more statements that represents
|
||||
/// a program.
|
||||
#[derive(Debug, PartialEq, Eq, Clone)]
|
||||
pub struct Ast {
|
||||
pub prog: Vec<Stmt>,
|
||||
/// Bitwise Not
|
||||
BNot,
|
||||
|
||||
/// Logical Not
|
||||
LNot,
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Eq, Clone)]
|
||||
pub enum Stmt {
|
||||
/// Just a simple expression. This might be an assignment, a function call or a calculation.
|
||||
Expr(Expr),
|
||||
/// A variable declaration and assignment. (variable name, assigned value)
|
||||
Let(String, Expr),
|
||||
/// A while loop consisting of a condition and a body. (condition, body)
|
||||
While(Expr, Ast),
|
||||
/// A for loop consisting of an initialization declaration, a condition, an advancement and a
|
||||
/// body. ((variable name, initial value), condition, advancement, body)
|
||||
For((String, Expr), Expr, Expr, Ast),
|
||||
/// If statement consisting of a condition, a true_body and a false_body.
|
||||
/// (condition, true_body, false_body)
|
||||
If(Expr, Ast, Ast),
|
||||
/// Debug print the value of an expression (show the internal type together with the value)
|
||||
DbgPrint(Expr),
|
||||
/// Print the value of an expression
|
||||
Print(Expr),
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Eq, Clone)]
|
||||
pub enum Expr {
|
||||
pub enum Expression {
|
||||
/// Integer literal (64-bit)
|
||||
I64(i64),
|
||||
/// String literal
|
||||
Str(Rc<String>),
|
||||
/// Identifier (variable name)
|
||||
Ident(String),
|
||||
String(Rc<String>),
|
||||
/// Variable
|
||||
Var(String),
|
||||
/// Binary operation. Consists of type, left hand side and right hand side
|
||||
BinOp(BinOpType, Box<Expr>, Box<Expr>),
|
||||
/// Unary operation. Consists of type and the value that is operated on
|
||||
UnOp(UnOpType, Box<Expr>),
|
||||
}
|
||||
BinOp(BinOpType, Box<Expression>, Box<Expression>),
|
||||
/// Unary operation. Consists of type and operand
|
||||
UnOp(UnOpType, Box<Expression>),
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Eq, Clone)]
|
||||
pub struct Loop {
|
||||
/// The condition that determines if the loop should continue
|
||||
pub condition: Expression,
|
||||
/// This is executed after each loop to advance the condition variables
|
||||
pub advancement: Option<Expression>,
|
||||
/// The loop body that is executed each loop
|
||||
pub body: Ast,
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Eq, Clone)]
|
||||
pub struct If {
|
||||
/// The condition
|
||||
pub condition: Expression,
|
||||
/// The body that is executed when condition is true
|
||||
pub body_true: Ast,
|
||||
/// The if body that is executed when the condition is false
|
||||
pub body_false: Ast,
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Eq, Clone)]
|
||||
pub enum Statement {
|
||||
Expr(Expression),
|
||||
Loop(Loop),
|
||||
If(If),
|
||||
Print(Expression),
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Eq, Clone, Default)]
|
||||
pub struct Ast {
|
||||
pub prog: Vec<Statement>,
|
||||
}
|
||||
|
||||
impl BinOpType {
|
||||
/// Get the precedence for a binary operator. Higher value means the OP is stronger binding.
|
||||
/// For example Multiplication is stronger than addition, so Mul has higher precedence than Add.
|
||||
///
|
||||
/// The operator precedences are derived from the C language operator precedences. While not all
|
||||
/// C operators are included or the exact same, the precedence oder is the same.
|
||||
/// See: https://en.cppreference.com/w/c/language/operator_precedence
|
||||
|
||||
pub fn precedence(&self) -> u8 {
|
||||
match self {
|
||||
BinOpType::Declare => 0,
|
||||
BinOpType::Assign => 1,
|
||||
BinOpType::LOr => 2,
|
||||
BinOpType::LAnd => 3,
|
||||
BinOpType::BOr => 4,
|
||||
BinOpType::BXor => 5,
|
||||
BinOpType::BAnd => 6,
|
||||
BinOpType::EquEqu | BinOpType::NotEqu => 7,
|
||||
BinOpType::Less | BinOpType::LessEqu | BinOpType::Greater | BinOpType::GreaterEqu => 8,
|
||||
BinOpType::Shl | BinOpType::Shr => 9,
|
||||
BinOpType::Add | BinOpType::Sub => 10,
|
||||
BinOpType::Mul | BinOpType::Div | BinOpType::Mod => 11,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
197
src/bytecode.rs
197
src/bytecode.rs
@@ -1,197 +0,0 @@
|
||||
use std::collections::HashMap;
|
||||
|
||||
use crate::ast::{Ast, Expr, Stmt, BinOpType};
|
||||
|
||||
pub mod op {
|
||||
type OpSize = u32;
|
||||
|
||||
pub const PUSH: OpSize = 0;
|
||||
pub const POP: OpSize = 1;
|
||||
pub const LOAD: OpSize = 2;
|
||||
pub const STORE: OpSize = 3;
|
||||
pub const ADD: OpSize = 4;
|
||||
pub const SUB: OpSize = 5;
|
||||
pub const MUL: OpSize = 6;
|
||||
pub const DIV: OpSize = 7;
|
||||
pub const MOD: OpSize = 8;
|
||||
pub const BOR: OpSize = 9;
|
||||
pub const BAND: OpSize = 10;
|
||||
pub const BXOR: OpSize = 11;
|
||||
pub const SHL: OpSize = 12;
|
||||
pub const SHR: OpSize = 13;
|
||||
pub const EQ: OpSize = 14;
|
||||
pub const NEQ: OpSize = 15;
|
||||
pub const GT: OpSize = 16;
|
||||
pub const GE: OpSize = 17;
|
||||
pub const LT: OpSize = 18;
|
||||
pub const LE: OpSize = 19;
|
||||
pub const JUMP: OpSize = 20;
|
||||
pub const JUMP_TRUE: OpSize = 21;
|
||||
pub const JUMP_FALSE: OpSize = 22;
|
||||
pub const PRINT: OpSize = 23;
|
||||
pub const DBG_PRINT: OpSize = 24;
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
pub struct Compiler {
|
||||
ops: Vec<u32>,
|
||||
global_vars: HashMap<String, u16>,
|
||||
}
|
||||
|
||||
impl Compiler {
|
||||
pub fn new() -> Self {
|
||||
Compiler::default()
|
||||
}
|
||||
|
||||
pub fn compile(&mut self, ast: &Ast) {
|
||||
for stmt in &ast.prog {
|
||||
match stmt {
|
||||
Stmt::Expr(expr) => {
|
||||
self.compile_expr(expr);
|
||||
self.ops.push(op::POP);
|
||||
}
|
||||
Stmt::Let(name, rhs) => {
|
||||
let id = self.global_vars.len() as u16;
|
||||
self.global_vars.insert(name.clone(), id);
|
||||
|
||||
self.compile_expr(rhs);
|
||||
self.gen_store(id);
|
||||
}
|
||||
Stmt::While(cond, body) => {
|
||||
self.compile_expr(cond);
|
||||
|
||||
self.ops.push(op::JUMP_FALSE);
|
||||
let idx_jmp = self.ops.len();
|
||||
self.gen_i64(0);
|
||||
|
||||
let idx_start = self.ops.len();
|
||||
|
||||
self.compile(body);
|
||||
|
||||
// check condition before loop jump
|
||||
self.compile_expr(cond);
|
||||
|
||||
self.ops.push(op::JUMP_TRUE);
|
||||
self.gen_i64(idx_start as i64);
|
||||
|
||||
self.overwrite_i64(idx_jmp, self.ops.len() as i64);
|
||||
}
|
||||
Stmt::For(_, _, _, _) => todo!(),
|
||||
Stmt::If(cond, if_block, else_block) => {
|
||||
self.compile_expr(cond);
|
||||
|
||||
self.ops.push(op::JUMP_FALSE);
|
||||
let idx_if = self.ops.len();
|
||||
self.gen_i64(0);
|
||||
|
||||
self.compile(if_block);
|
||||
|
||||
self.ops.push(op::JUMP);
|
||||
let idx_else = self.ops.len();
|
||||
self.gen_i64(0);
|
||||
|
||||
self.overwrite_i64(idx_if, self.ops.len() as i64);
|
||||
|
||||
self.compile(else_block);
|
||||
|
||||
self.overwrite_i64(idx_else, self.ops.len() as i64);
|
||||
|
||||
},
|
||||
Stmt::DbgPrint(expr) => {
|
||||
self.compile_expr(expr);
|
||||
self.ops.push(op::DBG_PRINT);
|
||||
}
|
||||
Stmt::Print(expr) => {
|
||||
self.compile_expr(expr);
|
||||
self.ops.push(op::PRINT);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn into_ops(self) -> Vec<u32> {
|
||||
self.ops
|
||||
}
|
||||
|
||||
pub fn compile_expr(&mut self, expr: &Expr) {
|
||||
match expr {
|
||||
Expr::I64(val) => {
|
||||
self.ops.push(op::PUSH);
|
||||
self.gen_i64(*val)
|
||||
}
|
||||
Expr::Ident(name) => {
|
||||
match self.global_vars.get(name).copied() {
|
||||
Some(addr) => self.gen_load(addr),
|
||||
None => panic!("Variable '{}' used before declaration", name),
|
||||
}
|
||||
},
|
||||
Expr::BinOp(bo, lhs, rhs) => self.compile_binop(bo, lhs, rhs),
|
||||
Expr::UnOp(_, _) => todo!(),
|
||||
Expr::Str(_) => todo!(),
|
||||
}
|
||||
}
|
||||
|
||||
fn compile_binop(&mut self, bo: &BinOpType, lhs: &Expr, rhs: &Expr) {
|
||||
|
||||
if matches!(bo, BinOpType::Assign) {
|
||||
self.compile_expr(rhs);
|
||||
|
||||
if let Expr::Ident(name) = lhs {
|
||||
let addr = *self.global_vars.get(name).expect("Trying to assign var before decl");
|
||||
self.gen_store(addr);
|
||||
} else {
|
||||
panic!("Trying to assign value to rvalue");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
self.compile_expr(lhs);
|
||||
self.compile_expr(rhs);
|
||||
|
||||
match bo {
|
||||
BinOpType::Add => self.ops.push(op::ADD),
|
||||
BinOpType::Sub => self.ops.push(op::SUB),
|
||||
BinOpType::Mul => self.ops.push(op::MUL),
|
||||
BinOpType::Div => self.ops.push(op::DIV),
|
||||
BinOpType::Mod => self.ops.push(op::MOD),
|
||||
BinOpType::BOr => self.ops.push(op::BOR),
|
||||
BinOpType::BAnd => self.ops.push(op::BAND),
|
||||
BinOpType::BXor => self.ops.push(op::BXOR),
|
||||
BinOpType::Shl => self.ops.push(op::SHL),
|
||||
BinOpType::Shr => self.ops.push(op::SHR),
|
||||
BinOpType::Equ => self.ops.push(op::EQ),
|
||||
BinOpType::Neq => self.ops.push(op::NEQ),
|
||||
BinOpType::Gt => self.ops.push(op::GT),
|
||||
BinOpType::Ge => self.ops.push(op::GE),
|
||||
BinOpType::Lt => self.ops.push(op::LT),
|
||||
BinOpType::Le => self.ops.push(op::LE),
|
||||
BinOpType::Assign => unreachable!(),
|
||||
}
|
||||
}
|
||||
|
||||
fn gen_i64(&mut self, val: i64) {
|
||||
self.ops.push((val & u32::MAX as i64) as u32);
|
||||
self.ops.push((val >> 32) as u32);
|
||||
}
|
||||
|
||||
fn overwrite_i64(&mut self, idx: usize, val: i64) {
|
||||
self.ops[idx] = (val & u32::MAX as i64) as u32;
|
||||
self.ops[idx+1] = (val >> 32) as u32;
|
||||
}
|
||||
|
||||
fn gen_load(&mut self, addr: u16) {
|
||||
self.ops.push(op::LOAD | (addr << 8) as u32);
|
||||
// self.gen_i64(addr as i64)
|
||||
}
|
||||
|
||||
fn gen_store(&mut self, addr: u16) {
|
||||
self.ops.push(op::STORE | (addr << 8) as u32);
|
||||
// self.gen_i64(addr as i64)
|
||||
}
|
||||
}
|
||||
|
||||
pub fn compile(ast: &Ast) -> Vec<u32> {
|
||||
let mut compiler = Compiler::new();
|
||||
compiler.compile(ast);
|
||||
compiler.into_ops()
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
use std::{collections::HashMap, fmt::Display, rc::Rc};
|
||||
use std::{fmt::Display, rc::Rc};
|
||||
|
||||
use crate::{
|
||||
ast::{Ast, BinOpType, Expr, Stmt, UnOpType},
|
||||
ast::{Ast, BinOpType, Expression, If, Statement, UnOpType},
|
||||
lexer::lex,
|
||||
parser::parse,
|
||||
};
|
||||
@@ -9,126 +9,143 @@ use crate::{
|
||||
#[derive(Debug, PartialEq, Eq, Clone)]
|
||||
pub enum Value {
|
||||
I64(i64),
|
||||
Str(Rc<String>),
|
||||
String(Rc<String>),
|
||||
}
|
||||
|
||||
pub struct Interpreter {
|
||||
/// The variable table maps all variables by their names to their values
|
||||
vartable: HashMap<String, Value>,
|
||||
// Variable table stores the runtime values of variables
|
||||
vartable: Vec<(String, Value)>,
|
||||
}
|
||||
|
||||
impl Interpreter {
|
||||
pub fn new() -> Self {
|
||||
let vartable = HashMap::new();
|
||||
Self { vartable }
|
||||
Self {
|
||||
vartable: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn run_text(&mut self, code: &str, print_tokens: bool, print_ast: bool) {
|
||||
let tokens = lex(code);
|
||||
fn get_var(&self, name: &str) -> Option<Value> {
|
||||
self.vartable
|
||||
.iter()
|
||||
.rev()
|
||||
.find(|it| it.0 == name)
|
||||
.map(|it| it.1.clone())
|
||||
}
|
||||
|
||||
fn get_var_mut(&mut self, name: &str) -> Option<&mut Value> {
|
||||
self.vartable
|
||||
.iter_mut()
|
||||
.rev()
|
||||
.find(|it| it.0 == name)
|
||||
.map(|it| &mut it.1)
|
||||
}
|
||||
|
||||
pub fn run_str(&mut self, code: &str, print_tokens: bool, print_ast: bool) {
|
||||
let tokens = lex(code).unwrap();
|
||||
if print_tokens {
|
||||
println!("Tokens: {:?}", tokens);
|
||||
}
|
||||
|
||||
let ast = parse(tokens);
|
||||
if print_ast {
|
||||
println!("Ast:\n{:#?}", ast);
|
||||
println!("{:#?}", ast);
|
||||
}
|
||||
|
||||
self.run(&ast);
|
||||
}
|
||||
|
||||
pub fn run(&mut self, prog: &Ast) {
|
||||
let vartable_len = self.vartable.len();
|
||||
for stmt in &prog.prog {
|
||||
match stmt {
|
||||
Stmt::Expr(expr) => {
|
||||
Statement::Expr(expr) => {
|
||||
self.resolve_expr(expr);
|
||||
}
|
||||
Stmt::DbgPrint(expr) => {
|
||||
let result = self.resolve_expr(expr);
|
||||
println!("{:?}", result);
|
||||
|
||||
Statement::Loop(looop) => {
|
||||
// loop runs as long condition != 0
|
||||
loop {
|
||||
if matches!(self.resolve_expr(&looop.condition), Value::I64(0)) {
|
||||
break;
|
||||
}
|
||||
|
||||
self.run(&looop.body);
|
||||
|
||||
if let Some(adv) = &looop.advancement {
|
||||
self.resolve_expr(&adv);
|
||||
}
|
||||
}
|
||||
}
|
||||
Stmt::Print(expr) => {
|
||||
|
||||
Statement::Print(expr) => {
|
||||
let result = self.resolve_expr(expr);
|
||||
print!("{}", result);
|
||||
}
|
||||
Stmt::Let(name, rhs) => {
|
||||
let result = self.resolve_expr(rhs);
|
||||
self.vartable.insert(name.clone(), result);
|
||||
}
|
||||
Stmt::For(init, condition, advance, body) => {
|
||||
// Execute initital let instruction
|
||||
let init_val = self.resolve_expr(&init.1);
|
||||
self.vartable.insert(init.0.clone(), init_val);
|
||||
|
||||
loop {
|
||||
// Check condition
|
||||
match self.resolve_expr(condition) {
|
||||
Value::I64(val) if val == 0 => break,
|
||||
Value::I64(_) => (),
|
||||
|
||||
Value::Str(text) if text.is_empty() => break,
|
||||
Value::Str(_) => (),
|
||||
}
|
||||
|
||||
// Execute loop body
|
||||
self.run(body);
|
||||
|
||||
// Execute advancement
|
||||
self.resolve_expr(advance);
|
||||
}
|
||||
}
|
||||
Stmt::While(condition, body) => {
|
||||
loop {
|
||||
// Check condition
|
||||
match self.resolve_expr(condition) {
|
||||
Value::I64(val) if val == 0 => break,
|
||||
Value::I64(_) => (),
|
||||
|
||||
Value::Str(text) if text.is_empty() => break,
|
||||
Value::Str(_) => (),
|
||||
}
|
||||
|
||||
// Execute loop body
|
||||
self.run(body);
|
||||
}
|
||||
}
|
||||
Stmt::If(condition, body_if, body_else) => {
|
||||
Statement::If(If {
|
||||
condition,
|
||||
body_true,
|
||||
body_false,
|
||||
}) => {
|
||||
if matches!(self.resolve_expr(condition), Value::I64(0)) {
|
||||
self.run(body_else);
|
||||
self.run(body_false);
|
||||
} else {
|
||||
self.run(body_if);
|
||||
self.run(body_true);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
self.vartable.truncate(vartable_len);
|
||||
}
|
||||
|
||||
fn resolve_expr(&mut self, expr: &Expr) -> Value {
|
||||
fn resolve_expr(&mut self, expr: &Expression) -> Value {
|
||||
match expr {
|
||||
Expr::I64(val) => Value::I64(*val),
|
||||
Expr::Str(name) => Value::Str(name.clone()),
|
||||
Expr::BinOp(bo, lhs, rhs) => self.resolve_binop(bo, &lhs, &rhs),
|
||||
Expr::UnOp(uo, val) => self.resolve_unop(uo, &val),
|
||||
Expr::Ident(name) => match self.vartable.get(name) {
|
||||
None => panic!("Runtime error: Use of undeclared variable '{}'", name),
|
||||
Some(val) => val.clone(),
|
||||
},
|
||||
Expression::I64(val) => Value::I64(*val),
|
||||
Expression::String(text) => Value::String(text.clone()),
|
||||
Expression::BinOp(bo, lhs, rhs) => self.resolve_binop(bo, lhs, rhs),
|
||||
Expression::UnOp(uo, operand) => self.resolve_unop(uo, operand),
|
||||
Expression::Var(name) => self.resolve_var(name),
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_binop(&mut self, bo: &BinOpType, lhs: &Expr, rhs: &Expr) -> Value {
|
||||
// Treat assignment separate from the other expressions
|
||||
if matches!(bo, BinOpType::Assign) {
|
||||
match lhs {
|
||||
Expr::Ident(name) => {
|
||||
let rhs = self.resolve_expr(rhs);
|
||||
self.vartable.get_mut(name).map(|var| *var = rhs.clone());
|
||||
return rhs;
|
||||
}
|
||||
_ => panic!("Runtime error: Left hand side of assignment must be an identifier"),
|
||||
fn resolve_var(&mut self, name: &str) -> Value {
|
||||
match self.get_var(name) {
|
||||
Some(val) => val.clone(),
|
||||
None => panic!("Variable '{}' used but not declared", name),
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_unop(&mut self, uo: &UnOpType, operand: &Expression) -> Value {
|
||||
let operand = self.resolve_expr(operand);
|
||||
|
||||
match (operand, uo) {
|
||||
(Value::I64(val), UnOpType::Negate) => Value::I64(-val),
|
||||
(Value::I64(val), UnOpType::BNot) => Value::I64(!val),
|
||||
(Value::I64(val), UnOpType::LNot) => Value::I64(if val == 0 { 1 } else { 0 }),
|
||||
_ => panic!("Value type is not compatible with unary operation"),
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_binop(&mut self, bo: &BinOpType, lhs: &Expression, rhs: &Expression) -> Value {
|
||||
let rhs = self.resolve_expr(rhs);
|
||||
|
||||
match (&bo, &lhs) {
|
||||
(BinOpType::Declare, Expression::Var(name)) => {
|
||||
self.vartable.push((name.clone(), rhs.clone()));
|
||||
return rhs;
|
||||
}
|
||||
(BinOpType::Assign, Expression::Var(name)) => {
|
||||
match self.get_var_mut(name) {
|
||||
Some(val) => *val = rhs.clone(),
|
||||
None => panic!("Runtime Error: Trying to assign value to undeclared variable"),
|
||||
}
|
||||
return rhs;
|
||||
}
|
||||
_ => (),
|
||||
}
|
||||
|
||||
let lhs = self.resolve_expr(lhs);
|
||||
let rhs = self.resolve_expr(rhs);
|
||||
|
||||
match (lhs, rhs) {
|
||||
(Value::I64(lhs), Value::I64(rhs)) => match bo {
|
||||
@@ -140,36 +157,29 @@ impl Interpreter {
|
||||
BinOpType::BOr => Value::I64(lhs | rhs),
|
||||
BinOpType::BAnd => Value::I64(lhs & rhs),
|
||||
BinOpType::BXor => Value::I64(lhs ^ rhs),
|
||||
BinOpType::LAnd => Value::I64(if (lhs != 0) && (rhs != 0) { 1 } else { 0 }),
|
||||
BinOpType::LOr => Value::I64(if (lhs != 0) || (rhs != 0) { 1 } else { 0 }),
|
||||
BinOpType::Shr => Value::I64(lhs >> rhs),
|
||||
BinOpType::Shl => Value::I64(lhs << rhs),
|
||||
BinOpType::Equ => Value::I64(if lhs == rhs { 1 } else { 0 }),
|
||||
BinOpType::Neq => Value::I64(if lhs != rhs { 1 } else { 0 }),
|
||||
BinOpType::Gt => Value::I64(if lhs > rhs { 1 } else { 0 }),
|
||||
BinOpType::Ge => Value::I64(if lhs >= rhs { 1 } else { 0 }),
|
||||
BinOpType::Lt => Value::I64(if lhs < rhs { 1 } else { 0 }),
|
||||
BinOpType::Le => Value::I64(if lhs <= rhs { 1 } else { 0 }),
|
||||
BinOpType::Assign => unreachable!(),
|
||||
BinOpType::EquEqu => Value::I64(if lhs == rhs { 1 } else { 0 }),
|
||||
BinOpType::NotEqu => Value::I64(if lhs != rhs { 1 } else { 0 }),
|
||||
BinOpType::Less => Value::I64(if lhs < rhs { 1 } else { 0 }),
|
||||
BinOpType::LessEqu => Value::I64(if lhs <= rhs { 1 } else { 0 }),
|
||||
BinOpType::Greater => Value::I64(if lhs > rhs { 1 } else { 0 }),
|
||||
BinOpType::GreaterEqu => Value::I64(if lhs >= rhs { 1 } else { 0 }),
|
||||
|
||||
BinOpType::Declare | BinOpType::Assign => unreachable!(),
|
||||
},
|
||||
_ => panic!("Value types are not compatible"),
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_unop(&mut self, uo: &UnOpType, val: &Expr) -> Value {
|
||||
let val = self.resolve_expr(val);
|
||||
match val {
|
||||
Value::I64(val) => match uo {
|
||||
UnOpType::Neg => Value::I64(-val),
|
||||
},
|
||||
_ => panic!("Invalid unary operation for type"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Display for Value {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Value::I64(val) => write!(f, "{}", val),
|
||||
Value::Str(text) => write!(f, "{}", text),
|
||||
Value::String(text) => write!(f, "{}", text),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -177,21 +187,26 @@ impl Display for Value {
|
||||
#[cfg(test)]
|
||||
mod test {
|
||||
use super::{Interpreter, Value};
|
||||
use crate::ast::{BinOpType, Expr};
|
||||
use crate::ast::{BinOpType, Expression};
|
||||
|
||||
#[test]
|
||||
fn test_interpreter_expr() {
|
||||
// Expression: 1 + 2 * 3 + 4
|
||||
// With precedence: (1 + (2 * 3)) + 4
|
||||
let ast = Expr::BinOp(
|
||||
let ast = Expression::BinOp(
|
||||
BinOpType::Add,
|
||||
Expr::BinOp(
|
||||
Expression::BinOp(
|
||||
BinOpType::Add,
|
||||
Expr::I64(1).into(),
|
||||
Expr::BinOp(BinOpType::Mul, Expr::I64(2).into(), Expr::I64(3).into()).into(),
|
||||
Expression::I64(1).into(),
|
||||
Expression::BinOp(
|
||||
BinOpType::Mul,
|
||||
Expression::I64(2).into(),
|
||||
Expression::I64(3).into(),
|
||||
)
|
||||
.into(),
|
||||
)
|
||||
.into(),
|
||||
Expr::I64(4).into(),
|
||||
Expression::I64(4).into(),
|
||||
);
|
||||
|
||||
let expected = Value::I64(11);
|
||||
|
||||
250
src/lexer.rs
250
src/lexer.rs
@@ -1,14 +1,31 @@
|
||||
use crate::token::Token;
|
||||
use anyhow::Result;
|
||||
use std::{iter::Peekable, str::Chars};
|
||||
use thiserror::Error;
|
||||
|
||||
use crate::token::{Keyword, Literal, Token};
|
||||
#[derive(Debug, Error)]
|
||||
pub enum LexErr {
|
||||
#[error("Failed to parse '{0}' as i64")]
|
||||
NumericParse(String),
|
||||
|
||||
#[error("Invalid escape character '\\{0}'")]
|
||||
InvalidStrEscape(char),
|
||||
|
||||
#[error("Lexer encountered unexpected char: '{0}'")]
|
||||
UnexpectedChar(char),
|
||||
|
||||
#[error("Missing closing string quote '\"'")]
|
||||
MissingClosingString,
|
||||
}
|
||||
|
||||
/// Lex the provided code into a Token Buffer
|
||||
pub fn lex(code: &str) -> Vec<Token> {
|
||||
pub fn lex(code: &str) -> Result<Vec<Token>, LexErr> {
|
||||
let mut lexer = Lexer::new(code);
|
||||
lexer.lex()
|
||||
}
|
||||
|
||||
struct Lexer<'a> {
|
||||
/// The sourcecode text as an iterator over the chars
|
||||
code: Peekable<Chars<'a>>,
|
||||
}
|
||||
|
||||
@@ -18,29 +35,21 @@ impl<'a> Lexer<'a> {
|
||||
Self { code }
|
||||
}
|
||||
|
||||
/// Advance to next character and return the removed char. If there is no next char, '\0'
|
||||
/// is returned.
|
||||
fn next(&mut self) -> char {
|
||||
self.code.next().unwrap_or('\0')
|
||||
}
|
||||
|
||||
/// Get the next character without removing it. If there is no next char, '\0' is returned.
|
||||
fn peek(&mut self) -> char {
|
||||
self.code.peek().copied().unwrap_or('\0')
|
||||
}
|
||||
|
||||
fn lex(&mut self) -> Vec<Token> {
|
||||
fn lex(&mut self) -> Result<Vec<Token>, LexErr> {
|
||||
let mut tokens = Vec::new();
|
||||
|
||||
loop {
|
||||
match self.next() {
|
||||
// End of text
|
||||
// Stop lexing at EOF
|
||||
'\0' => break,
|
||||
|
||||
// Skip whitespace
|
||||
' ' | '\r' | '\n' | '\t' => (),
|
||||
' ' | '\t' | '\n' | '\r' => (),
|
||||
|
||||
// Handle tokens that span two characters
|
||||
// Line comment. Consume every char until linefeed (next line)
|
||||
'/' if matches!(self.peek(), '/') => while !matches!(self.next(), '\n' | '\0') {},
|
||||
|
||||
// Double character tokens
|
||||
'>' if matches!(self.peek(), '>') => {
|
||||
self.next();
|
||||
tokens.push(Token::Shr);
|
||||
@@ -51,26 +60,35 @@ impl<'a> Lexer<'a> {
|
||||
}
|
||||
'=' if matches!(self.peek(), '=') => {
|
||||
self.next();
|
||||
tokens.push(Token::Equ);
|
||||
tokens.push(Token::EquEqu);
|
||||
}
|
||||
'!' if matches!(self.peek(), '=') => {
|
||||
self.next();
|
||||
tokens.push(Token::Neq);
|
||||
tokens.push(Token::NotEqu);
|
||||
}
|
||||
'<' if matches!(self.peek(), '=') => {
|
||||
self.next();
|
||||
tokens.push(Token::Le);
|
||||
tokens.push(Token::LAngleEqu);
|
||||
}
|
||||
'>' if matches!(self.peek(), '=') => {
|
||||
self.next();
|
||||
tokens.push(Token::Ge);
|
||||
tokens.push(Token::RAngleEqu);
|
||||
}
|
||||
'$' if matches!(self.peek(), '$') => {
|
||||
'<' if matches!(self.peek(), '-') => {
|
||||
self.next();
|
||||
tokens.push(Token::DoubleDollar);
|
||||
tokens.push(Token::LArrow);
|
||||
}
|
||||
'&' if matches!(self.peek(), '&') => {
|
||||
self.next();
|
||||
tokens.push(Token::LAnd);
|
||||
}
|
||||
'|' if matches!(self.peek(), '|') => {
|
||||
self.next();
|
||||
tokens.push(Token::LOr);
|
||||
}
|
||||
|
||||
// Handle tokens that span one character
|
||||
// Single character tokens
|
||||
';' => tokens.push(Token::Semicolon),
|
||||
'+' => tokens.push(Token::Add),
|
||||
'-' => tokens.push(Token::Sub),
|
||||
'*' => tokens.push(Token::Mul),
|
||||
@@ -81,34 +99,36 @@ impl<'a> Lexer<'a> {
|
||||
'^' => tokens.push(Token::BXor),
|
||||
'(' => tokens.push(Token::LParen),
|
||||
')' => tokens.push(Token::RParen),
|
||||
'<' => tokens.push(Token::Lt),
|
||||
'>' => tokens.push(Token::Gt),
|
||||
'=' => tokens.push(Token::Assign),
|
||||
';' => tokens.push(Token::Semicolon),
|
||||
'{' => tokens.push(Token::LBrace),
|
||||
'}' => tokens.push(Token::RBrace),
|
||||
'$' => tokens.push(Token::Dollar),
|
||||
'~' => tokens.push(Token::Tilde),
|
||||
'<' => tokens.push(Token::LAngle),
|
||||
'>' => tokens.push(Token::RAngle),
|
||||
'=' => tokens.push(Token::Equ),
|
||||
'{' => tokens.push(Token::LBraces),
|
||||
'}' => tokens.push(Token::RBraces),
|
||||
'!' => tokens.push(Token::LNot),
|
||||
|
||||
// Handle special multicharacter tokens
|
||||
// Special tokens with variable length
|
||||
|
||||
// Lex numbers
|
||||
ch @ '0'..='9' => tokens.push(self.lex_number(ch)),
|
||||
// Lex multiple characters together as numbers
|
||||
ch @ '0'..='9' => tokens.push(self.lex_number(ch)?),
|
||||
|
||||
// Lex strings
|
||||
'"' => tokens.push(self.lex_string()),
|
||||
// Lex multiple characters together as a string
|
||||
'"' => tokens.push(self.lex_str()?),
|
||||
|
||||
// Lex identifiers
|
||||
ch @ ('a'..='z' | 'A'..='Z' | '_') => tokens.push(self.lex_ident(ch)),
|
||||
// Lex multiple characters together as identifier
|
||||
ch @ ('a'..='z' | 'A'..='Z' | '_') => tokens.push(self.lex_identifier(ch)?),
|
||||
|
||||
// Any other character is unexpected
|
||||
ch => panic!("Lexer encountered unexpected char: '{}'", ch),
|
||||
ch => Err(LexErr::UnexpectedChar(ch))?,
|
||||
}
|
||||
}
|
||||
|
||||
tokens
|
||||
Ok(tokens)
|
||||
}
|
||||
|
||||
fn lex_number(&mut self, first_char: char) -> Token {
|
||||
/// Lex multiple characters as a number until encountering a non numeric digit. This includes
|
||||
/// the first character
|
||||
fn lex_number(&mut self, first_char: char) -> Result<Token, LexErr> {
|
||||
// String representation of the integer value
|
||||
let mut sval = String::from(first_char);
|
||||
|
||||
// Do as long as a next char exists and it is a numeric char
|
||||
@@ -127,101 +147,105 @@ impl<'a> Lexer<'a> {
|
||||
}
|
||||
}
|
||||
|
||||
// TODO: We only added numeric chars to the string, but the conversion could still fail
|
||||
Token::Literal(Literal::I64(sval.parse().unwrap()))
|
||||
// Try to convert the string representation of the value to i64
|
||||
let i64val = sval.parse().map_err(|_| LexErr::NumericParse(sval))?;
|
||||
Ok(Token::I64(i64val))
|
||||
}
|
||||
|
||||
/// Lex an identifier from the character stream. The first char has to have been consumed
|
||||
/// from the stream already and is passed as an argument instead.
|
||||
fn lex_ident(&mut self, first_char: char) -> Token {
|
||||
let mut ident = String::from(first_char);
|
||||
/// Lex characters as a string until encountering an unescaped closing doublequoute char '"'
|
||||
fn lex_str(&mut self) -> Result<Token, LexErr> {
|
||||
// Opening " was consumed in match
|
||||
|
||||
// Do as long as a next char exists and it is a valid ident char
|
||||
while let 'a'..='z' | 'A'..='Z' | '_' | '0'..='9' = self.peek() {
|
||||
// The next char is verified to be Some, so unwrap is safe
|
||||
ident.push(self.next());
|
||||
}
|
||||
|
||||
// Check if the identifier is a keyword
|
||||
match ident.as_str() {
|
||||
"true" => Token::Literal(Literal::I64(1)),
|
||||
"false" => Token::Literal(Literal::I64(0)),
|
||||
"let" => Token::Keyword(Keyword::Let),
|
||||
"while" => Token::Keyword(Keyword::While),
|
||||
"if" => Token::Keyword(Keyword::If),
|
||||
"else" => Token::Keyword(Keyword::Else),
|
||||
"for" => Token::Keyword(Keyword::For),
|
||||
|
||||
_ => Token::Ident(ident),
|
||||
}
|
||||
}
|
||||
|
||||
/// Lex a string token from the character stream. This requires the initial quote '"' to be
|
||||
/// consumed before.
|
||||
fn lex_string(&mut self) -> Token {
|
||||
let mut text = String::new();
|
||||
|
||||
let mut escape = false;
|
||||
|
||||
// Do as long as a next char exists and it is not '"'
|
||||
// Read all chars until encountering the closing "
|
||||
loop {
|
||||
if escape {
|
||||
escape = false;
|
||||
|
||||
// Escape characters
|
||||
match self.next() {
|
||||
'\\' => text.push('\\'),
|
||||
'n' => text.push('\n'),
|
||||
'r' => text.push('\r'),
|
||||
't' => text.push('\t'),
|
||||
ch => panic!("Invalid string escape: '{:?}'", ch),
|
||||
}
|
||||
} else {
|
||||
match self.peek() {
|
||||
// Doublequote '"' ends the string lexing
|
||||
'"' => {
|
||||
self.next();
|
||||
break;
|
||||
}
|
||||
// Backslash '\' escapes the next character
|
||||
'\\' => {
|
||||
self.next();
|
||||
escape = true;
|
||||
}
|
||||
|
||||
// Reached end of text but didn't encounter closing doublequote '"'
|
||||
'\0' => panic!("String is never terminated (missing '\"')"),
|
||||
|
||||
_ => text.push(self.next()),
|
||||
}
|
||||
match self.peek() {
|
||||
'"' => break,
|
||||
// If the end of file is reached while still waiting for '"', error out
|
||||
'\0' => Err(LexErr::MissingClosingString)?,
|
||||
_ => match self.next() {
|
||||
// Backshlash indicates an escaped character
|
||||
'\\' => match self.next() {
|
||||
'n' => text.push('\n'),
|
||||
'r' => text.push('\r'),
|
||||
't' => text.push('\t'),
|
||||
'\\' => text.push('\\'),
|
||||
'"' => text.push('"'),
|
||||
ch => Err(LexErr::InvalidStrEscape(ch))?,
|
||||
},
|
||||
// All other characters are simply appended to the string
|
||||
ch => text.push(ch),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
Token::Literal(Literal::Str(text))
|
||||
// Consume closing "
|
||||
self.next();
|
||||
|
||||
Ok(Token::String(text))
|
||||
}
|
||||
|
||||
/// Lex characters from the text as an identifier. This includes the first character passed in
|
||||
fn lex_identifier(&mut self, first_char: char) -> Result<Token, LexErr> {
|
||||
let mut ident = String::from(first_char);
|
||||
|
||||
// Do as long as a next char exists and it is a valid char for an identifier
|
||||
loop {
|
||||
match self.peek() {
|
||||
// In the middle of an identifier numbers are also allowed
|
||||
'a'..='z' | 'A'..='Z' | '0'..='9' | '_' => {
|
||||
ident.push(self.next());
|
||||
}
|
||||
// Next char is not valid, so stop and finish the ident token
|
||||
_ => break,
|
||||
}
|
||||
}
|
||||
|
||||
// Check for pre-defined keywords
|
||||
let token = match ident.as_str() {
|
||||
"loop" => Token::Loop,
|
||||
"print" => Token::Print,
|
||||
"if" => Token::If,
|
||||
"else" => Token::Else,
|
||||
|
||||
// If it doesn't match a keyword, it is a normal identifier
|
||||
_ => Token::Ident(ident),
|
||||
};
|
||||
|
||||
Ok(token)
|
||||
}
|
||||
|
||||
/// Advance to next character and return the removed char
|
||||
fn next(&mut self) -> char {
|
||||
self.code.next().unwrap_or('\0')
|
||||
}
|
||||
|
||||
/// Get the next character without removing it
|
||||
fn peek(&mut self) -> char {
|
||||
self.code.peek().copied().unwrap_or('\0')
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use crate::token::Literal;
|
||||
|
||||
use super::{lex, Token};
|
||||
|
||||
#[test]
|
||||
fn test_lexer() {
|
||||
let code = "33 +5*2 + 4456467*2334+3 % - / << ^ | & >>";
|
||||
let expected = vec![
|
||||
Token::Literal(Literal::I64(33)),
|
||||
Token::I64(33),
|
||||
Token::Add,
|
||||
Token::Literal(Literal::I64(5)),
|
||||
Token::I64(5),
|
||||
Token::Mul,
|
||||
Token::Literal(Literal::I64(2)),
|
||||
Token::I64(2),
|
||||
Token::Add,
|
||||
Token::Literal(Literal::I64(4456467)),
|
||||
Token::I64(4456467),
|
||||
Token::Mul,
|
||||
Token::Literal(Literal::I64(2334)),
|
||||
Token::I64(2334),
|
||||
Token::Add,
|
||||
Token::Literal(Literal::I64(3)),
|
||||
Token::I64(3),
|
||||
Token::Mod,
|
||||
Token::Sub,
|
||||
Token::Div,
|
||||
@@ -232,7 +256,7 @@ mod tests {
|
||||
Token::Shr,
|
||||
];
|
||||
|
||||
let actual = lex(code);
|
||||
let actual = lex(code).unwrap();
|
||||
assert_eq!(expected, actual);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
pub mod lexer;
|
||||
pub mod parser;
|
||||
pub mod interpreter;
|
||||
pub mod token;
|
||||
pub mod parser;
|
||||
pub mod ast;
|
||||
pub mod bytecode;
|
||||
pub mod vm;
|
||||
pub mod interpreter;
|
||||
|
||||
48
src/main.rs
48
src/main.rs
@@ -1,6 +1,10 @@
|
||||
use std::{env::args, io::Write};
|
||||
use std::{
|
||||
env::args,
|
||||
fs,
|
||||
io::{stdin, stdout, Write},
|
||||
};
|
||||
|
||||
use nek_lang::{interpreter::Interpreter, lexer::lex, parser::parse, bytecode::compile, vm::Vm};
|
||||
use nek_lang::interpreter::Interpreter;
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct CliConfig {
|
||||
@@ -11,53 +15,41 @@ struct CliConfig {
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let mut cfg = CliConfig::default();
|
||||
let mut conf = CliConfig::default();
|
||||
|
||||
// Go through all commandline arguments except the first (filename)
|
||||
for arg in args().skip(1) {
|
||||
match arg.as_str() {
|
||||
"--tokens" | "-t" => cfg.print_tokens = true,
|
||||
"--ast" | "-a" => cfg.print_ast = true,
|
||||
"--interactive" | "-i" => cfg.interactive = true,
|
||||
file if cfg.file.is_none() => cfg.file = Some(file.to_string()),
|
||||
"--token" | "-t" => conf.print_tokens = true,
|
||||
"--ast" | "-a" => conf.print_ast = true,
|
||||
"--interactive" | "-i" => conf.interactive = true,
|
||||
file if conf.file.is_none() => conf.file = Some(file.to_string()),
|
||||
_ => panic!("Invalid argument: '{}'", arg),
|
||||
}
|
||||
}
|
||||
|
||||
let mut interpreter = Interpreter::new();
|
||||
|
||||
if let Some(file) = &cfg.file {
|
||||
let code = std::fs::read_to_string(file).expect(&format!("File not found: '{}'", file));
|
||||
let tokens = lex(&code);
|
||||
let ast = parse(tokens);
|
||||
|
||||
let prog = compile(&ast);
|
||||
|
||||
// println!("{:?}", prog);
|
||||
|
||||
let mut vm = Vm::new(prog);
|
||||
|
||||
vm.run();
|
||||
|
||||
// interpreter.run_text(&code, cfg.print_tokens, cfg.print_ast);
|
||||
if let Some(file) = &conf.file {
|
||||
let code = fs::read_to_string(file).expect(&format!("File not found: '{}'", file));
|
||||
interpreter.run_str(&code, conf.print_tokens, conf.print_ast);
|
||||
}
|
||||
|
||||
if cfg.interactive || cfg.file.is_none() {
|
||||
|
||||
if conf.interactive || conf.file.is_none() {
|
||||
let mut code = String::new();
|
||||
|
||||
loop {
|
||||
print!(">> ");
|
||||
std::io::stdout().flush().unwrap();
|
||||
stdout().flush().unwrap();
|
||||
|
||||
code.clear();
|
||||
std::io::stdin().read_line(&mut code).unwrap();
|
||||
let code = code.trim();
|
||||
stdin().read_line(&mut code).unwrap();
|
||||
|
||||
if code == "exit" {
|
||||
if code.trim() == "exit" {
|
||||
break;
|
||||
}
|
||||
|
||||
interpreter.run_text(&code, cfg.print_tokens, cfg.print_ast);
|
||||
interpreter.run_str(&code, conf.print_tokens, conf.print_ast);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
325
src/parser.rs
325
src/parser.rs
@@ -1,9 +1,13 @@
|
||||
use std::iter::Peekable;
|
||||
|
||||
use crate::{
|
||||
ast::{Ast, BinOpType, Expr, Stmt, UnOpType},
|
||||
token::{Keyword, Literal, Token},
|
||||
};
|
||||
use crate::ast::*;
|
||||
use crate::token::Token;
|
||||
|
||||
/// Parse the given tokens into an abstract syntax tree
|
||||
pub fn parse<T: Iterator<Item = Token>, A: IntoIterator<IntoIter = T>>(tokens: A) -> Ast {
|
||||
let mut parser = Parser::new(tokens);
|
||||
parser.parse()
|
||||
}
|
||||
|
||||
struct Parser<T: Iterator<Item = Token>> {
|
||||
tokens: Peekable<T>,
|
||||
@@ -16,171 +20,149 @@ impl<T: Iterator<Item = Token>> Parser<T> {
|
||||
Self { tokens }
|
||||
}
|
||||
|
||||
/// Get the next Token without removing it
|
||||
fn peek(&mut self) -> &Token {
|
||||
self.tokens.peek().unwrap_or(&Token::EoF)
|
||||
}
|
||||
|
||||
/// Advance to next Token and return the removed Token
|
||||
fn next(&mut self) -> Token {
|
||||
self.tokens.next().unwrap_or(Token::EoF)
|
||||
}
|
||||
|
||||
/// Parse tokens into an abstract syntax tree. This will continuously parse statements until
|
||||
/// encountering end-of-file or a block end '}' .
|
||||
fn parse(&mut self) -> Ast {
|
||||
let mut prog = Vec::new();
|
||||
|
||||
loop {
|
||||
let stmt = match self.peek() {
|
||||
match self.peek() {
|
||||
Token::Semicolon => {
|
||||
self.next();
|
||||
continue;
|
||||
}
|
||||
Token::EoF => break,
|
||||
Token::RBrace => break,
|
||||
Token::EoF | Token::RBraces => break,
|
||||
|
||||
Token::Keyword(keyword) => match keyword {
|
||||
Keyword::Let => self.parse_let_stmt(),
|
||||
Keyword::While => self.parse_while(),
|
||||
Keyword::If => self.parse_if(),
|
||||
Keyword::For => self.parse_for(),
|
||||
Keyword::Else => panic!("Unexpected else keyword"),
|
||||
},
|
||||
|
||||
Token::Dollar => {
|
||||
self.next();
|
||||
Stmt::Print(self.parse_expr())
|
||||
}
|
||||
Token::DoubleDollar => {
|
||||
self.next();
|
||||
Stmt::DbgPrint(self.parse_expr())
|
||||
}
|
||||
// By default try to parse an expression
|
||||
_ => Stmt::Expr(self.parse_expr()),
|
||||
};
|
||||
|
||||
prog.push(stmt);
|
||||
// By default try to lex a statement
|
||||
_ => prog.push(self.parse_stmt()),
|
||||
}
|
||||
}
|
||||
|
||||
Ast { prog }
|
||||
}
|
||||
|
||||
fn parse_for(&mut self) -> Stmt {
|
||||
if !matches!(self.next(), Token::Keyword(Keyword::For)) {
|
||||
panic!("Error parsing for: Expected for token");
|
||||
/// Parse a single statement from the tokens.
|
||||
fn parse_stmt(&mut self) -> Statement {
|
||||
match self.peek() {
|
||||
Token::Loop => Statement::Loop(self.parse_loop()),
|
||||
|
||||
Token::Print => {
|
||||
self.next();
|
||||
|
||||
let expr = self.parse_expr();
|
||||
|
||||
// After a statement, there must be a semicolon
|
||||
if !matches!(self.next(), Token::Semicolon) {
|
||||
panic!("Expected semicolon after statement");
|
||||
}
|
||||
|
||||
Statement::Print(expr)
|
||||
}
|
||||
|
||||
Token::If => Statement::If(self.parse_if()),
|
||||
|
||||
// If it is not a loop, try to lex as an expression
|
||||
_ => {
|
||||
let stmt = Statement::Expr(self.parse_expr());
|
||||
|
||||
// After a statement, there must be a semicolon
|
||||
if !matches!(self.next(), Token::Semicolon) {
|
||||
panic!("Expected semicolon after statement");
|
||||
}
|
||||
|
||||
stmt
|
||||
}
|
||||
}
|
||||
|
||||
let init = match self.parse_let_stmt() {
|
||||
Stmt::Let(name, rhs) => (name, rhs),
|
||||
_ => unreachable!(),
|
||||
};
|
||||
|
||||
if !matches!(self.next(), Token::Semicolon) {
|
||||
panic!("Error parsing for: Expected semicolon token");
|
||||
}
|
||||
|
||||
let condition = self.parse_expr();
|
||||
|
||||
if !matches!(self.next(), Token::Semicolon) {
|
||||
panic!("Error parsing for: Expected semicolon token");
|
||||
}
|
||||
|
||||
let advance = self.parse_expr();
|
||||
|
||||
if !matches!(self.next(), Token::LBrace) {
|
||||
panic!("Error parsing for: Expected '{{' token");
|
||||
}
|
||||
|
||||
let body = self.parse();
|
||||
|
||||
if !matches!(self.next(), Token::RBrace) {
|
||||
panic!("Error parsing for: Expected '}}' token");
|
||||
}
|
||||
|
||||
Stmt::For(init, condition, advance, body)
|
||||
}
|
||||
|
||||
fn parse_if(&mut self) -> Stmt {
|
||||
if !matches!(self.next(), Token::Keyword(Keyword::If)) {
|
||||
panic!("Error parsing if: Expected if token");
|
||||
/// Parse an if statement from the tokens
|
||||
fn parse_if(&mut self) -> If {
|
||||
if !matches!(self.next(), Token::If) {
|
||||
panic!("Error lexing if: Expected if token");
|
||||
}
|
||||
|
||||
let condition = self.parse_expr();
|
||||
|
||||
if !matches!(self.next(), Token::LBrace) {
|
||||
panic!("Error parsing if: Expected '{{' token");
|
||||
if !matches!(self.next(), Token::LBraces) {
|
||||
panic!("Error lexing if: Expected '{{'")
|
||||
}
|
||||
|
||||
let body_if = self.parse();
|
||||
let body_true = self.parse();
|
||||
|
||||
if !matches!(self.next(), Token::RBrace) {
|
||||
panic!("Error parsing if: Expected '}}' token");
|
||||
if !matches!(self.next(), Token::RBraces) {
|
||||
panic!("Error lexing if: Expected '}}'")
|
||||
}
|
||||
|
||||
let mut body_else = Ast { prog: Vec::new() };
|
||||
let mut body_false = Ast::default();
|
||||
|
||||
if matches!(self.peek(), Token::Keyword(Keyword::Else)) {
|
||||
if matches!(self.peek(), Token::Else) {
|
||||
self.next();
|
||||
|
||||
if !matches!(self.next(), Token::LBrace) {
|
||||
panic!("Error parsing else: Expected '{{' token");
|
||||
if !matches!(self.next(), Token::LBraces) {
|
||||
panic!("Error lexing if: Expected '{{'")
|
||||
}
|
||||
|
||||
body_else = self.parse();
|
||||
body_false = self.parse();
|
||||
|
||||
if !matches!(self.next(), Token::RBrace) {
|
||||
panic!("Error parsing else: Expected '}}' token");
|
||||
if !matches!(self.next(), Token::RBraces) {
|
||||
panic!("Error lexing if: Expected '}}'")
|
||||
}
|
||||
}
|
||||
|
||||
Stmt::If(condition, body_if, body_else)
|
||||
If {
|
||||
condition,
|
||||
body_true,
|
||||
body_false,
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_while(&mut self) -> Stmt {
|
||||
if !matches!(self.next(), Token::Keyword(Keyword::While)) {
|
||||
panic!("Error parsing while: Expected while token");
|
||||
/// Parse a loop statement from the tokens
|
||||
fn parse_loop(&mut self) -> Loop {
|
||||
if !matches!(self.next(), Token::Loop) {
|
||||
panic!("Error lexing loop: Expected loop token");
|
||||
}
|
||||
|
||||
let condition = self.parse_expr();
|
||||
let mut advancement = None;
|
||||
|
||||
if !matches!(self.next(), Token::LBrace) {
|
||||
panic!("Error parsing while: Expected '{{' token");
|
||||
let body;
|
||||
|
||||
match self.next() {
|
||||
Token::LBraces => {
|
||||
body = self.parse();
|
||||
}
|
||||
|
||||
Token::Semicolon => {
|
||||
advancement = Some(self.parse_expr());
|
||||
|
||||
if !matches!(self.next(), Token::LBraces) {
|
||||
panic!("Error lexing loop: Expected '{{'")
|
||||
}
|
||||
|
||||
body = self.parse();
|
||||
}
|
||||
|
||||
_ => panic!("Error lexing loop: Expected ';' or '{{'"),
|
||||
}
|
||||
|
||||
let body = self.parse();
|
||||
|
||||
if !matches!(self.next(), Token::RBrace) {
|
||||
panic!("Error parsing while: Expected '}}' token");
|
||||
if !matches!(self.next(), Token::RBraces) {
|
||||
panic!("Error lexing loop: Expected '}}'")
|
||||
}
|
||||
|
||||
Stmt::While(condition, body)
|
||||
Loop {
|
||||
condition,
|
||||
advancement,
|
||||
body,
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_let_stmt(&mut self) -> Stmt {
|
||||
if !matches!(self.next(), Token::Keyword(Keyword::Let)) {
|
||||
panic!("Error parsing let: Expected let token");
|
||||
}
|
||||
|
||||
let name = match self.next() {
|
||||
Token::Ident(name) => name,
|
||||
_ => panic!("Error parsing let: Expected identifier after let"),
|
||||
};
|
||||
|
||||
if !matches!(self.next(), Token::Assign) {
|
||||
panic!("Error parsing let: Expected assignment token");
|
||||
}
|
||||
|
||||
let rhs = self.parse_expr();
|
||||
|
||||
Stmt::Let(name, rhs)
|
||||
}
|
||||
|
||||
fn parse_expr(&mut self) -> Expr {
|
||||
/// Parse a single expression from the tokens
|
||||
fn parse_expr(&mut self) -> Expression {
|
||||
let lhs = self.parse_primary();
|
||||
self.parse_expr_precedence(lhs, 0)
|
||||
}
|
||||
|
||||
/// Parse binary expressions with a precedence equal to or higher than min_prec
|
||||
fn parse_expr_precedence(&mut self, mut lhs: Expr, min_prec: u8) -> Expr {
|
||||
fn parse_expr_precedence(&mut self, mut lhs: Expression, min_prec: u8) -> Expression {
|
||||
while let Some(binop) = &self.peek().try_to_binop() {
|
||||
// Stop if the next operator has a lower binding power
|
||||
if !(binop.precedence() >= min_prec) {
|
||||
@@ -201,75 +183,74 @@ impl<T: Iterator<Item = Token>> Parser<T> {
|
||||
rhs = self.parse_expr_precedence(rhs, binop.precedence() + 1);
|
||||
}
|
||||
|
||||
lhs = Expr::BinOp(binop, lhs.into(), rhs.into());
|
||||
lhs = Expression::BinOp(binop, lhs.into(), rhs.into());
|
||||
}
|
||||
|
||||
lhs
|
||||
}
|
||||
|
||||
/// Parse a primary expression (for now only number)
|
||||
fn parse_primary(&mut self) -> Expr {
|
||||
fn parse_primary(&mut self) -> Expression {
|
||||
match self.next() {
|
||||
Token::Literal(Literal::I64(val)) => Expr::I64(val),
|
||||
// Literal i64
|
||||
Token::I64(val) => Expression::I64(val),
|
||||
|
||||
Token::Literal(Literal::Str(text)) => Expr::Str(text.into()),
|
||||
// Literal String
|
||||
Token::String(text) => Expression::String(text.into()),
|
||||
|
||||
Token::Ident(name) => Expr::Ident(name),
|
||||
Token::Ident(name) => Expression::Var(name),
|
||||
|
||||
// Parentheses grouping
|
||||
Token::LParen => {
|
||||
// The tokens was an opening parenthesis, so parse a full expression again as the
|
||||
// expression inside the parentheses `"(" expr ")"`
|
||||
let inner = self.parse_expr();
|
||||
let inner_expr = self.parse_expr();
|
||||
|
||||
// If there is no closing parenthesis after the expression, it is a syntax error
|
||||
// Verify that there is a closing parenthesis
|
||||
if !matches!(self.next(), Token::RParen) {
|
||||
panic!("Error parsing primary expr: Missing closing parenthesis ')'");
|
||||
panic!("Error parsing primary expr: Exepected closing parenthesis ')'");
|
||||
}
|
||||
|
||||
inner
|
||||
inner_expr
|
||||
}
|
||||
|
||||
Token::Sub => Expr::UnOp(UnOpType::Neg, self.parse_primary().into()),
|
||||
// Unary negation
|
||||
Token::Sub => {
|
||||
let operand = self.parse_primary();
|
||||
Expression::UnOp(UnOpType::Negate, operand.into())
|
||||
}
|
||||
|
||||
// Unary bitwise not (bitflip)
|
||||
Token::Tilde => {
|
||||
let operand = self.parse_primary();
|
||||
Expression::UnOp(UnOpType::BNot, operand.into())
|
||||
}
|
||||
|
||||
// Unary logical not
|
||||
Token::LNot => {
|
||||
let operand = self.parse_primary();
|
||||
Expression::UnOp(UnOpType::LNot, operand.into())
|
||||
}
|
||||
|
||||
tok => panic!("Error parsing primary expr: Unexpected Token '{:?}'", tok),
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
/// Get the next Token without removing it
|
||||
fn peek(&mut self) -> &Token {
|
||||
self.tokens.peek().unwrap_or(&Token::EoF)
|
||||
}
|
||||
|
||||
pub fn parse<T: Iterator<Item = Token>, A: IntoIterator<IntoIter = T>>(tokens: A) -> Ast {
|
||||
let mut parser = Parser::new(tokens);
|
||||
parser.parse()
|
||||
}
|
||||
|
||||
impl BinOpType {
|
||||
/// Get the precedence for a binary operator. Higher value means the OP is stronger binding.
|
||||
/// For example Multiplication is stronger than addition, so Mul has higher precedence than Add.
|
||||
///
|
||||
/// The operator precedences are derived from the C language operator precedences. While not all
|
||||
/// C operators are included or the exact same, the precedence oder is the same.
|
||||
/// See: https://en.cppreference.com/w/c/language/operator_precedence
|
||||
fn precedence(&self) -> u8 {
|
||||
match self {
|
||||
BinOpType::Assign => 0,
|
||||
BinOpType::BOr => 1,
|
||||
BinOpType::BXor => 2,
|
||||
BinOpType::BAnd => 3,
|
||||
BinOpType::Equ | BinOpType::Neq => 4,
|
||||
BinOpType::Gt | BinOpType::Ge | BinOpType::Lt | BinOpType::Le => 5,
|
||||
BinOpType::Shl | BinOpType::Shr => 6,
|
||||
BinOpType::Add | BinOpType::Sub => 7,
|
||||
BinOpType::Mul | BinOpType::Div | BinOpType::Mod => 8,
|
||||
}
|
||||
/// Advance to next Token and return the removed Token
|
||||
fn next(&mut self) -> Token {
|
||||
self.tokens.next().unwrap_or(Token::EoF)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{parse, BinOpType, Expr};
|
||||
use super::{parse, BinOpType, Expression};
|
||||
use crate::{
|
||||
parser::{Ast, Stmt},
|
||||
token::{Literal, Token},
|
||||
parser::{Ast, Statement},
|
||||
token::Token,
|
||||
};
|
||||
|
||||
#[test]
|
||||
@@ -277,28 +258,34 @@ mod tests {
|
||||
// Expression: 1 + 2 * 3 + 4
|
||||
// With precedence: (1 + (2 * 3)) + 4
|
||||
let tokens = [
|
||||
Token::Literal(Literal::I64(1)),
|
||||
Token::I64(1),
|
||||
Token::Add,
|
||||
Token::Literal(Literal::I64(2)),
|
||||
Token::I64(2),
|
||||
Token::Mul,
|
||||
Token::Literal(Literal::I64(3)),
|
||||
Token::I64(3),
|
||||
Token::Sub,
|
||||
Token::Literal(Literal::I64(4)),
|
||||
Token::I64(4),
|
||||
Token::Semicolon,
|
||||
];
|
||||
|
||||
let expected = Expr::BinOp(
|
||||
let expected = Statement::Expr(Expression::BinOp(
|
||||
BinOpType::Sub,
|
||||
Expr::BinOp(
|
||||
Expression::BinOp(
|
||||
BinOpType::Add,
|
||||
Expr::I64(1).into(),
|
||||
Expr::BinOp(BinOpType::Mul, Expr::I64(2).into(), Expr::I64(3).into()).into(),
|
||||
Expression::I64(1).into(),
|
||||
Expression::BinOp(
|
||||
BinOpType::Mul,
|
||||
Expression::I64(2).into(),
|
||||
Expression::I64(3).into(),
|
||||
)
|
||||
.into(),
|
||||
)
|
||||
.into(),
|
||||
Expr::I64(4).into(),
|
||||
);
|
||||
Expression::I64(4).into(),
|
||||
));
|
||||
|
||||
let expected = Ast {
|
||||
prog: vec![Stmt::Expr(expected)],
|
||||
prog: vec![expected],
|
||||
};
|
||||
|
||||
let actual = parse(tokens);
|
||||
|
||||
135
src/token.rs
135
src/token.rs
@@ -1,63 +1,39 @@
|
||||
use crate::ast::BinOpType;
|
||||
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub enum Literal {
|
||||
pub enum Token {
|
||||
/// Integer literal (64-bit)
|
||||
I64(i64),
|
||||
|
||||
/// String literal ("Some string")
|
||||
Str(String),
|
||||
}
|
||||
/// String literal
|
||||
String(String),
|
||||
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub enum Keyword {
|
||||
/// Let identifier (let)
|
||||
Let,
|
||||
|
||||
/// While (while)
|
||||
While,
|
||||
|
||||
/// For (for)
|
||||
For,
|
||||
|
||||
/// If (if)
|
||||
If,
|
||||
|
||||
/// Else (else)
|
||||
Else,
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub enum Token {
|
||||
/// Literal values
|
||||
Literal(Literal),
|
||||
|
||||
/// Identifier (variable / function / ... name)
|
||||
/// Identifier (name for variables, functions, ...)
|
||||
Ident(String),
|
||||
|
||||
/// Specific identifiers that have a special meaning as keywords
|
||||
Keyword(Keyword),
|
||||
/// Loop keyword (loop)
|
||||
Loop,
|
||||
|
||||
/// Left parenthesis ('(')
|
||||
/// Print keyword (print)
|
||||
Print,
|
||||
|
||||
/// If keyword (if)
|
||||
If,
|
||||
|
||||
/// Else keyword (else)
|
||||
Else,
|
||||
|
||||
/// Left Parenthesis ('(')
|
||||
LParen,
|
||||
|
||||
/// Right parentheses (')')
|
||||
/// Right Parenthesis (')')
|
||||
RParen,
|
||||
|
||||
/// Left brace ({)
|
||||
LBrace,
|
||||
/// Left curly braces ({)
|
||||
LBraces,
|
||||
|
||||
/// Right brace (})
|
||||
RBrace,
|
||||
|
||||
/// Dollar sign ($)
|
||||
Dollar,
|
||||
|
||||
/// Double Dollar sign ($$)
|
||||
DoubleDollar,
|
||||
|
||||
/// Assignment (single equal) (=)
|
||||
Assign,
|
||||
/// Right curly braces (})
|
||||
RBraces,
|
||||
|
||||
/// Plus (+)
|
||||
Add,
|
||||
@@ -74,6 +50,12 @@ pub enum Token {
|
||||
/// Percent (%)
|
||||
Mod,
|
||||
|
||||
/// Equal Equal (==)
|
||||
EquEqu,
|
||||
|
||||
/// Exclamationmark Equal (!=)
|
||||
NotEqu,
|
||||
|
||||
/// Pipe (|)
|
||||
BOr,
|
||||
|
||||
@@ -83,30 +65,42 @@ pub enum Token {
|
||||
/// Circumflex (^)
|
||||
BXor,
|
||||
|
||||
/// Logical AND (&&)
|
||||
LAnd,
|
||||
|
||||
/// Logical OR (||)
|
||||
LOr,
|
||||
|
||||
/// Shift Left (<<)
|
||||
Shl,
|
||||
|
||||
/// Shift Right (>>)
|
||||
Shr,
|
||||
|
||||
/// Equal sign (==)
|
||||
/// Tilde (~)
|
||||
Tilde,
|
||||
|
||||
/// Logical not (!)
|
||||
LNot,
|
||||
|
||||
/// Left angle bracket (<)
|
||||
LAngle,
|
||||
|
||||
/// Right angle bracket (>)
|
||||
RAngle,
|
||||
|
||||
/// Left angle bracket Equal (<=)
|
||||
LAngleEqu,
|
||||
|
||||
/// Left angle bracket Equal (>=)
|
||||
RAngleEqu,
|
||||
|
||||
/// Left arrow (<-)
|
||||
LArrow,
|
||||
|
||||
/// Equal Sign (=)
|
||||
Equ,
|
||||
|
||||
/// Not Equal sign (!=)
|
||||
Neq,
|
||||
|
||||
/// Greater than (>)
|
||||
Gt,
|
||||
|
||||
/// Greater or equal (>=)
|
||||
Ge,
|
||||
|
||||
/// Less than (<)
|
||||
Lt,
|
||||
|
||||
/// Less or equal (<=)
|
||||
Le,
|
||||
|
||||
/// Semicolon (;)
|
||||
Semicolon,
|
||||
|
||||
@@ -128,18 +122,23 @@ impl Token {
|
||||
Token::BOr => BinOpType::BOr,
|
||||
Token::BXor => BinOpType::BXor,
|
||||
|
||||
Token::LAnd => BinOpType::LAnd,
|
||||
Token::LOr => BinOpType::LOr,
|
||||
|
||||
Token::Shl => BinOpType::Shl,
|
||||
Token::Shr => BinOpType::Shr,
|
||||
|
||||
Token::Equ => BinOpType::Equ,
|
||||
Token::Neq => BinOpType::Neq,
|
||||
Token::EquEqu => BinOpType::EquEqu,
|
||||
Token::NotEqu => BinOpType::NotEqu,
|
||||
|
||||
Token::Gt => BinOpType::Gt,
|
||||
Token::Ge => BinOpType::Ge,
|
||||
Token::Lt => BinOpType::Lt,
|
||||
Token::Le => BinOpType::Le,
|
||||
Token::LAngle => BinOpType::Less,
|
||||
Token::LAngleEqu => BinOpType::LessEqu,
|
||||
|
||||
Token::Assign => BinOpType::Assign,
|
||||
Token::RAngle => BinOpType::Greater,
|
||||
Token::RAngleEqu => BinOpType::GreaterEqu,
|
||||
|
||||
Token::LArrow => BinOpType::Declare,
|
||||
Token::Equ => BinOpType::Assign,
|
||||
|
||||
_ => return None,
|
||||
})
|
||||
|
||||
208
src/vm.rs
208
src/vm.rs
@@ -1,208 +0,0 @@
|
||||
use crate::{bytecode::op::*, interpreter::Value};
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
pub struct Vm {
|
||||
prog: Vec<u32>,
|
||||
ip: usize,
|
||||
stack: Vec<Value>,
|
||||
|
||||
/// This isn't actually a heap. It's actually still more of a f*cked up stack
|
||||
heap: Vec<Value>,
|
||||
}
|
||||
|
||||
macro_rules! binop_stack {
|
||||
($self:ident, $op:tt) => {
|
||||
{
|
||||
let rhs = $self.stack.pop().unwrap();
|
||||
let lhs = $self.stack.last_mut().unwrap();
|
||||
match (lhs, rhs) {
|
||||
(Value::I64(lhs), Value::I64(rhs)) => *lhs = *lhs $op rhs,
|
||||
_ => panic!("Invalid data for add"),
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
impl Vm {
|
||||
pub fn new(prog: Vec<u32>) -> Self {
|
||||
Self {
|
||||
prog,
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
|
||||
pub fn run(&mut self) {
|
||||
while let Some(op) = self.prog.get(self.ip).copied() {
|
||||
self.ip += 1;
|
||||
|
||||
match op & 0xff {
|
||||
PUSH => {
|
||||
let val = self.read_i64();
|
||||
self.stack.push(Value::I64(val));
|
||||
}
|
||||
POP => {
|
||||
self.stack.pop();
|
||||
}
|
||||
LOAD => {
|
||||
// let addr = self.read_i64() as usize;
|
||||
let addr = (op >> 8) as usize;
|
||||
|
||||
if let Some(val) = self.heap.get(addr) {
|
||||
self.stack.push(val.clone());
|
||||
} else {
|
||||
panic!("Trying to load from uninitialized heap");
|
||||
}
|
||||
}
|
||||
STORE => {
|
||||
let val = self
|
||||
.stack
|
||||
.pop()
|
||||
.expect("Trying to pop value from stack for storing");
|
||||
// let addr = self.read_i64() as usize;
|
||||
let addr = (op >> 8) as usize;
|
||||
|
||||
if self.heap.len() == addr {
|
||||
self.heap.push(val);
|
||||
} else {
|
||||
self.heap[addr] = val;
|
||||
}
|
||||
}
|
||||
PRINT => {
|
||||
let val = self
|
||||
.stack
|
||||
.pop()
|
||||
.expect("Trying to pop value from stack for printing");
|
||||
print!("{}", val);
|
||||
}
|
||||
DBG_PRINT => {
|
||||
let val = self
|
||||
.stack
|
||||
.pop()
|
||||
.expect("Trying to pop value from stack for printing");
|
||||
print!("{:?}", val);
|
||||
}
|
||||
|
||||
ADD => {
|
||||
binop_stack!(self, +);
|
||||
// self.stack.push(Value::I64(vals.0 + vals.1))
|
||||
}
|
||||
SUB => {
|
||||
binop_stack!(self, -);
|
||||
// let vals = self.pop2_i64();
|
||||
// self.stack.push(Value::I64(vals.0 - vals.1))
|
||||
}
|
||||
MUL => {
|
||||
binop_stack!(self, *);
|
||||
// let vals = self.pop2_i64();
|
||||
// self.stack.push(Value::I64(vals.0 * vals.1))
|
||||
}
|
||||
DIV => {
|
||||
binop_stack!(self, /);
|
||||
// let vals = self.pop2_i64();
|
||||
// self.stack.push(Value::I64(vals.0 / vals.1))
|
||||
}
|
||||
MOD => {
|
||||
binop_stack!(self, %);
|
||||
// let vals = self.pop2_i64();
|
||||
// self.stack.push(Value::I64(vals.0 % vals.1))
|
||||
}
|
||||
EQ => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack
|
||||
.push(Value::I64(if vals.0 == vals.1 { 1 } else { 0 }))
|
||||
}
|
||||
NEQ => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack
|
||||
.push(Value::I64(if vals.0 != vals.1 { 1 } else { 0 }))
|
||||
}
|
||||
GT => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack
|
||||
.push(Value::I64(if vals.0 > vals.1 { 1 } else { 0 }))
|
||||
}
|
||||
GE => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack
|
||||
.push(Value::I64(if vals.0 >= vals.1 { 1 } else { 0 }))
|
||||
}
|
||||
LT => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack
|
||||
.push(Value::I64(if vals.0 < vals.1 { 1 } else { 0 }))
|
||||
}
|
||||
LE => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack
|
||||
.push(Value::I64(if vals.0 <= vals.1 { 1 } else { 0 }))
|
||||
}
|
||||
BOR => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack.push(Value::I64(vals.0 | vals.1))
|
||||
}
|
||||
BAND => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack.push(Value::I64(vals.0 & vals.1))
|
||||
}
|
||||
BXOR => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack.push(Value::I64(vals.0 ^ vals.1))
|
||||
}
|
||||
SHL => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack.push(Value::I64(vals.0 << vals.1))
|
||||
}
|
||||
SHR => {
|
||||
let vals = self.pop2_i64();
|
||||
self.stack.push(Value::I64(vals.0 >> vals.1))
|
||||
}
|
||||
JUMP => {
|
||||
self.ip = self.read_i64() as usize;
|
||||
}
|
||||
JUMP_TRUE => {
|
||||
let jmp_target = self.read_i64() as usize;
|
||||
if !matches!(self.stack.pop(), Some(Value::I64(0))) {
|
||||
self.ip = jmp_target;
|
||||
}
|
||||
}
|
||||
JUMP_FALSE => {
|
||||
let jmp_target = self.read_i64() as usize;
|
||||
if matches!(self.stack.pop(), Some(Value::I64(0))) {
|
||||
self.ip = jmp_target;
|
||||
}
|
||||
}
|
||||
_ => panic!("Invalid opcode")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn pop2_i64(&mut self) -> (i64, i64) {
|
||||
let rhs = self.stack.pop();
|
||||
let lhs = self.stack.pop();
|
||||
match (lhs, rhs) {
|
||||
(Some(Value::I64(lhs)), Some(Value::I64(rhs))) => (lhs, rhs),
|
||||
_ => panic!("Invalid data for add"),
|
||||
}
|
||||
}
|
||||
|
||||
fn read_i64(&mut self) -> i64 {
|
||||
let mut val = if let Some(val) = self.prog.get(self.ip).copied() {
|
||||
val
|
||||
} else {
|
||||
panic!("Expected Value as next OP")
|
||||
} as i64;
|
||||
|
||||
self.ip += 1;
|
||||
|
||||
val |= (if let Some(val) = self.prog.get(self.ip).copied() {
|
||||
val
|
||||
} else {
|
||||
panic!("Expected Value as next OP")
|
||||
} as i64)
|
||||
<< 32;
|
||||
|
||||
self.ip += 1;
|
||||
|
||||
val
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user